Help needed on Blob storage

CrateDB
1

We are planning to use crateDB as Blob storage.

I am trying to upload csv file and images.

I just refer the below guide and I could not upload any csv file or images. I can only upload plain text like a word or a sentence.

Can you please help me with uploading files and images?

Here is a repro, including the error message:

python3 -c 'import hashlib; print(hashlib.sha1("airport_code.csv".encode("utf-8")).hexdigest())'
curl -isSX  PUT '127.0.0.1:4200/_blobs/myblobs/c62c0e7f78f96d579609030606ad5c0ee0bd9b46' -d "@airport_code.csv"

Response:

HTTP/1.1 100 Continue

HTTP/1.1 400 Bad Request
content-length: 0
2

You might alternative use the crate-python library:

That blog post might be outdated

1 Like
3

Hi @Raghavendra_Hegde,

for uploading files, there are two differences compared to the examples from the documentation/blog which only deal with string literals.

SHA-1 hash
The Python example doesn’t actually load the file and hashes the content (it would only hash the file name). To run it on the file contents, the example needs to be changed to this:

python3 -c 'from pathlib import Path; import hashlib; print(hashlib.sha1(Path("airport_code.csv").read_text().encode("utf-8")).hexdigest())'

Alternatively, on a Unix command line, you can also run:

$ sha1sum airport_code.csv

Uploading via curl
To issue the corresponding HTTP PUT request, use the --data-binary parameter instead of --data (or short -d) like so:

curl -isSX  PUT '127.0.0.1:4200/_blobs/myblobs/1be168ff837f043bde17c0314341c84271047b31' --data-binary "@airport_code.csv"

Complete example workflow
Here is a complete example:

$ echo "123 test" > airport_code.csv
$ python3 -c 'from pathlib import Path; import hashlib; print(hashlib.sha1(Path("airport_code.csv").read_text().encode("utf-8")).hexdigest())'
449b02c475e7d7d6e3c1f366c9b0c1f6a5486924
$ sha1sum airport_code.csv 
449b02c475e7d7d6e3c1f366c9b0c1f6a5486924  airport_code.csv
$ curl -isSX  PUT '127.0.0.1:4200/_blobs/myblobs/449b02c475e7d7d6e3c1f366c9b0c1f6a5486924' --data-binary "@airport_code.csv" 
HTTP/1.1 201 Created
content-length: 0
$ curl -sS '127.0.0.1:4200/_blobs/myblobs/449b02c475e7d7d6e3c1f366c9b0c1f6a5486924'
123 test

Hope this helps!

4

This is working like a charm. Thank a lot @hammerhead