# How to upload a blob using curl?

**URL:** <https://community.cratedb.com/t/how-to-upload-a-blob-using-curl/1939>\
**Category:** CrateDB\
**Created:** [December 28, 2024, 8:12pm UTC](https://community.cratedb.com/t/how-to-upload-a-blob-using-curl/1939 "2024-12-28T20:12:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![MirtoBusico](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/mirtobusico/32/1322_2.png) [@MirtoBusico](https://community.cratedb.com/u/MirtoBusico)\
**Post date:** [December 28, 2024, 8:12pm UTC](https://community.cratedb.com/t/how-to-upload-a-blob-using-curl/1939/1 "2024-12-28T20:12:45Z")

</div>

I’m trying to upload a blob using curl; but I receive an authorization error.

Im following the example at [using blob tables - upload](https://cratedb.com/docs/crate/reference/en/latest/general/blobs.html#using-blob-tables)

The example code don’t use any kind of authentication

```auto
sh$ curl -isSX PUT '127.0.0.1:4200/_blobs/myblobs/4a756ca07e9487f482465a99e8286abc86ba4dc7' -d 'contents'
HTTP/1.1 201 Created
content-length: 0

```

I’m using host based authentication. I created CrateDB with crate-operator using this configuration file

```auto
apiVersion: cloud.crate.io/v1
kind: CrateDB
metadata:
  name: my-cluster
  namespace: for-crate
spec:
  cluster:
    imageRegistry: crate
    name: my-crate
    version: 5.8.1
    allowedCIDRs:
    - 192.168.151.0/24
    - 10.0.0.0/8
  nodes:
    data:
    - name: hot
      replicas: 3
      resources:
        limits:
          cpu: 4
          memory: 4Gi
        disk:
          count: 1
          size: 16GiB
          storageClass: longhorn
        heapRatio: 0.25

```

CrateDB answer on these addresses of the “192.168.151.0/24” network

```auto
service/crate-my-cluster LoadBalancer 10.43.2.208 192.168.151.21,192.168.151.22,192.168.151.23 4200:31111/TCP,5432:32328/TCP 8m42s

```

The machine issuing the command is on the same network

```auto
ip -c a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:bd:99:96 brd ff:ff:ff:ff:ff:ff
    inet 192.168.152.40/24 brd 192.168.152.255 scope global noprefixroute enp1s0
       valid_lft forever preferred_lft forever
3: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:f4:fe:e5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.151.40/24 brd 192.168.151.255 scope global noprefixroute enp2s0
       valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:b0:6b:a0:3f brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

```

From this machine using crate I see the attachments blob table

```auto
crash --verbose --hosts "192.168.151.21" -U system
+----------------------------+------------+---------+-----------+---------+
| server_url | node_name | version | connected | message |
+----------------------------+------------+---------+-----------+---------+
| http://192.168.151.21:4200 | data-hot-1 | 5.8.1 | TRUE | OK |
+----------------------------+------------+---------+-----------+---------+
CONNECT OK
CLUSTER CHECK OK
TYPES OF NODE CHECK OK
cr> select * from blob.attachments;
+--------+---------------+
| digest | last_modified |
+--------+---------------+
+--------+---------------+
SELECT 0 rows in set (0.023 sec)

```

(remembering to set the “system” user password in the environment variable “CRATEPW”)

Then I created a simple script to upload a file

```auto
sysop@h5a-dev:~/h5a/software/pcams/storehouse/cratedb/attachments$ cat upload 
#!/bin/bash

HASH=`sha1sum $1 |sed 's/ /|/' | awk -F '|' '{print $1}'`

echo ===== HASH ========
echo $HASH
echo =
echo ===== UPLOAD ======

curl -isSX PUT '192.168.151.21:4200/_blobs/attachments/$HASH' -d $1

sysop@h5a-dev:~/h5a/software/pcams/storehouse/cratedb/attachments$

```

But when using this script I receive the authorization error

```auto
sysop@h5a-dev:~/h5a/software/pcams/storehouse/cratedb/attachments$ ./upload ROG_CROSSHAIR_X670E_HERO_h732.png 
===== HASH ========
9a3993e2ecbbb3b1ea1435650cedd7fec2454e73
=
===== UPLOAD ======
HTTP/1.1 401 Unauthorized
content-length: 137
www-authenticate: Basic realm="CrateDB Authenticator"

No valid auth.host_based.config entry found for host "192.168.151.40", user "crate", protocol "http". Did you enable TLS in your client?
sysop@h5a-dev:~/h5a/software/pcams/storehouse/cratedb/attachments$ 

```

What I’m doing wrong?

**UPDATE**

Tried to add authorization in curl: same error but strange characters in the error message

New script

```auto

HASH=`sha1sum $1 |sed 's/ /|/' | awk -F '|' '{print $1}'`

echo ===== HASH ========
echo $HASH
echo =
echo ===== UPLOAD ======
echo =

curl -vvv -isSX PUT '192.168.151.21:4200/_blobs/attachments/$HASH' -H "Authorization: Basic S9xsloAiGVvZ7iqaDQjtAzasmiFedEYlc1ajapZWDikhPbZTkj" --data-binary @$1

echo ' '

```

The execution

```auto
sysop@h5a-dev:~/h5a/software/pcams/storehouse/cratedb/attachments$ ./upload ROG_CROSSHAIR_X670E_HERO_h732.png 
===== HASH ========
9a3993e2ecbbb3b1ea1435650cedd7fec2454e73
=
===== UPLOAD ======
=
* Trying 192.168.151.21:4200...
* Connected to 192.168.151.21 (192.168.151.21) port 4200 (#0)
> PUT /_blobs/attachments/$HASH HTTP/1.1
> Host: 192.168.151.21:4200
> User-Agent: curl/7.88.1
> Accept: */*
> Authorization: Basic S9xsloAiGVvZ7iqaDQjtAzasmiFedEYlc1ajapZWDikhPbZTkj
> Content-Length: 818987
> Content-Type: application/x-www-form-urlencoded
> 
< HTTP/1.1 401 Unauthorized
HTTP/1.1 401 Unauthorized
< content-length: 169
content-length: 169
< www-authenticate: Basic realm="CrateDB Authenticator"
www-authenticate: Basic realm="CrateDB Authenticator"

* HTTP error before end of send, stop sending
< 
* Excess found in a read: excess = 26, size = 169, maxdownload = 169, bytecount = 0
* Closing connection 0
�6��!^tF%sV�j�V)!=�S�", protocol "http". Did you e t "192.168.151.40", user "K�l��"[��*�
sysop@h5a-dev:~/h5a/software/pcams/storehouse/cratedb/attachments$ 

```

---

<div class="post-metadata">

**Author:** ![MirtoBusico](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/mirtobusico/32/1322_2.png) [@MirtoBusico](https://community.cratedb.com/u/MirtoBusico)\
**Post date:** [December 29, 2024, 8:29pm UTC](https://community.cratedb.com/t/how-to-upload-a-blob-using-curl/1939/2 "2024-12-29T20:29:13Z")

</div>

**UPDATE - SOME SUCCESS**

I gave up with the examples and tried with the curl manual.  
I was able to upload a file using the “ **–user username:password** ” and the “**–data-binary @[path to file]**” parameters

New script

```auto
#!/bin/bash

HASH=`sha1sum $1 |sed 's/ /|/' | awk -F '|' '{print $1}'`

echo ===== HASH ========
echo $HASH
echo =

echo ===== URL =========
URL=192.168.151.21:4200/_blobs/attachments/$HASH
echo $URL
echo =

echo ===== UPLOAD ======
echo =

curl -vvvv -isSX PUT $URL --user system:S9xsloAiGVvZ7iqaDQjtAzasmiFedEYlc1ajapZWDikhPbZTkj --data-binary @$1

echo ' '

```

The execution says it worked

```auto
sysop@h5a-dev:~/h5a/software/pcams/storehouse/cratedb/attachments$ ./upload ROG_CROSSHAIR_X670E_HERO_h732.png 
===== HASH ========
9a3993e2ecbbb3b1ea1435650cedd7fec2454e73
=
===== URL =========
192.168.151.21:4200/_blobs/attachments/9a3993e2ecbbb3b1ea1435650cedd7fec2454e73
=
===== UPLOAD ======
=
* Trying 192.168.151.21:4200...
* Connected to 192.168.151.21 (192.168.151.21) port 4200 (#0)
* Server auth using Basic with user 'system'
> PUT /_blobs/attachments/9a3993e2ecbbb3b1ea1435650cedd7fec2454e73 HTTP/1.1
> Host: 192.168.151.21:4200
> Authorization: Basic c3lzdGVtOlM5eHNsb0FpR1Z2WjdpcWFEUWp0QXphc21pRmVkRVlsYzFhamFwWldEaWtoUGJaVGtq
> User-Agent: curl/7.88.1
> Accept: */*
> Content-Length: 818987
> Content-Type: application/x-www-form-urlencoded
> 
* We are completely uploaded and fine
< HTTP/1.1 201 Created
HTTP/1.1 201 Created
< content-length: 0
content-length: 0

< 
* Connection #0 to host 192.168.151.21 left intact
 
sysop@h5a-dev:~/h5a/software/pcams/storehouse/cratedb/attachments$

```

Now I have to verify if the file was not corrupted during the upload

Any hint?

---

<div class="post-metadata">

**Author:** ![hernanc](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/hernanc/32/676_2.png) [@hernanc](https://community.cratedb.com/u/hernanc)\
**Post date:** [December 30, 2024, 8:33am UTC](https://community.cratedb.com/t/how-to-upload-a-blob-using-curl/1939/3 "2024-12-30T08:33:01Z")

</div>

> [@MirtoBusico](#):
>
> Now I have to verify if the file was not corrupted during the upload

I think that to confirm your approach worked correctly we could just download the file, calculate its hash, and then check if it matches the value you had before uploading.

---

<div class="post-metadata">

**Author:** ![MirtoBusico](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/mirtobusico/32/1322_2.png) [@MirtoBusico](https://community.cratedb.com/u/MirtoBusico)\
**Post date:** [December 30, 2024, 9:15am UTC](https://community.cratedb.com/t/how-to-upload-a-blob-using-curl/1939/4 "2024-12-30T09:15:27Z")

</div>

Yes, the downloaded file is identical to the uploaded one.

Problem solved… but I panicked (see my next post)