Help Needed: Storing Binary Files and Metadata in CrateDB Using NiFi


I have a few days of experience with CrateDB and NiFi. I would like to transfer a binary file (such as an image, video, or Excel file) using NiFi to CrateDB. I am unsure how to create the appropriate objects in CrateDB to store binary files in one table and metadata for these files in another table. Additionally, I am not certain how to build the correct process in NiFi.

Could someone provide some materials on this topic or give me some guidance on how to perform this operation?

Thank you in advance for your help.

Best regards.

1 Like

Thanks for reaching out and trying CrateDB.

For information on using CrateDB with NiFi please take a look at Connecting to CrateDB from Apache NiFi - Tutorials - CrateDB Community

Regarding the storage of binary files in CrateDB, you would probably be looking at Exploring Blob storage in CrateDB - Tutorials - CrateDB Community

If the files are small, there is also the option of Base64 encoding them and then storing them in a TEXT INDEX OFF STORAGE WITH (columnstore = false) column.

I hope this helps, do not hesitate to reach out again if you have any further questions.