Report bugs for Synology DSM deploy hook

[tresni]

Please report bugs for the Synology DSM hook here. Remember to include debug logs

acme.sh  --deploy --deploy-hook synology_dsm .....   --debug 2

[lippertmarkus]

@tresni thank's for your work! This doesn't support automatically renewing the cert, does it?

[tresni]

@lippertmarkus If you mean will the Synology automatically renew the certs, no. However, renewed certificates will be updated on the synology.

If you want to do renewals on your synology, I do this using a cronjob. Control Panel -> Task Scheduler -> Create -> Scheduled Task -> User-defined Script. Under General, set User to root. Under Schedule, set a schedule you feel is appropriate (I do monthly.) Under Task Settings, put the following in the Run Command box (assuming you installed to /usr/local/share/acme.sh):

/usr/local/share/acme.sh/acme.sh --cron --home /usr/local/share/acme.sh/

[lippertmarkus]

@tresni Thanks for the quick response. I do basically the same, see https://github.com/lippertmarkus/synology-le-dns-auto-renew
I just think searching for all the directories of all apps where the certificate is used like the script in the Wiki and my script does is a bit ugly.

When I saw that you use the web API in your deployment hook, I wondered if we could use the this API also for the renewing process?

[tresni]

This does both issuance and renewal. No weird directory handling, it just uses the API for everything. Maybe an example is better then trying to explain :)

ash-4.3# acme.sh --issue -d example.us --dns dns_cf
...
[Fri Mar  6 22:21:47 MST 2020] Your cert is in  /usr/local/share/acme.sh/example.us/example.us.cer
[Fri Mar  6 22:21:47 MST 2020] Your cert key is in  /usr/local/share/acme.sh/example.us/example.us.key
[Fri Mar  6 22:21:47 MST 2020] The intermediate CA cert is in  /usr/local/share/acme.sh/example.us/ca.cer
[Fri Mar  6 22:21:47 MST 2020] And the full chain certs is there:  /usr/local/share/acme.sh/example.us/fullchain.cer
ash-4.3# export SYNO_Username=... SYNO_Password=...
ash-4.3# SYNO_Create=1 SYNO_Certificate=example.us acme.sh -d example.us --deploy --deploy-hook synology_dsm
[Fri Mar  6 22:22:40 MST 2020] Logging into localhost:5000
[Fri Mar  6 22:22:47 MST 2020] Getting certificates in Synology DSM
[Fri Mar  6 22:22:47 MST 2020] Generate form POST request
[Fri Mar  6 22:22:47 MST 2020] Upload certificate to the Synology DSM
[Fri Mar  6 22:22:53 MST 2020] http services were NOT restarted
[Fri Mar  6 22:22:53 MST 2020] Success
ash-4.3# acme.sh -d example.us --renew --force
[Fri Mar  6 22:24:03 MST 2020] Renew: 'example.us'
[Fri Mar  6 22:24:04 MST 2020] Single domain='example.us'
[Fri Mar  6 22:24:04 MST 2020] Getting domain auth token for each domain
[Fri Mar  6 22:24:06 MST 2020] Getting webroot for domain='example.us'
[Fri Mar  6 22:24:06 MST 2020] example.us is already verified, skip dns-01.
[Fri Mar  6 22:24:06 MST 2020] Verify finished, start to sign.
[Fri Mar  6 22:24:06 MST 2020] Lets finalize the order, Le_OrderFinalize: https://acme-v02.api.letsencrypt.org/acme/finalize/.../...
[Fri Mar  6 22:24:07 MST 2020] Download cert, Le_LinkCert: https://acme-v02.api.letsencrypt.org/acme/cert/...
[Fri Mar  6 22:24:08 MST 2020] Cert success.
...
[Fri Mar  6 22:24:08 MST 2020] Your cert is in  /usr/local/share/acme.sh/example.us/example.us.cer
[Fri Mar  6 22:24:08 MST 2020] Your cert key is in  /usr/local/share/acme.sh/example.us/example.us.key
[Fri Mar  6 22:24:08 MST 2020] The intermediate CA cert is in  /usr/local/share/acme.sh/example.us/ca.cer
[Fri Mar  6 22:24:08 MST 2020] And the full chain certs is there:  /usr/local/share/acme.sh/example.us/fullchain.cer
[Fri Mar  6 22:24:09 MST 2020] Logging into localhost:5000
[Fri Mar  6 22:24:09 MST 2020] Getting certificates in Synology DSM
[Fri Mar  6 22:24:10 MST 2020] Generate form POST request
[Fri Mar  6 22:24:10 MST 2020] Upload certificate to the Synology DSM
[Fri Mar  6 22:24:10 MST 2020] http services were NOT restarted
[Fri Mar  6 22:24:10 MST 2020] Success
ash-4.3#

[lippertmarkus]

@tresni great, works like a charm! I added OTP/2-factor auth support in #2782 :)

[lippertmarkus]

@tresni sorry for hijacking this issue for discussing. Please tell me if you want to discuss elsewhere :)

While implementing #2782 I found that there is a binary which allows executing API requests without requiring authentication:

sudo /usr/syno/bin/synowebapi --exec api=SYNO.Core.Certificate.CRT method=list version=1
[Line 259] Exec WebAPI:  api=SYNO.Core.Certificate.CRT, version=1, method=list, param={}, runner=
{
   "data" : {
      "certificates" : [
         {
...

Maybe this could be used to implement a second way for importing the certificates? This wouldn't require any access information, so no scheme, host, port, username, password or device id environment variables. On the other hand this can only be used with a locally installed version of acme.sh and requires root. What do you think? Did you already try that?

I just briefly looked into that but couldn't figure out how to prepare a request for sending the binary certificate data with this method, but I think it should work somehow.

[tresni]

This module is designed where it can be run from any system, not just the Synology device itself, in line with what I believe acme.sh is attempting to accomplish. So using the WebApi allows for that, using the binary would not. Definitely a good find though!

[tatablack]

Nice work with the hook, much cleaner than looping over folders and certs. 👍 🙂

I was trying it today, but I ran into two separate issues:

Password
The password is not escaped when hitting login.cgi in this line, and mine (aA#99B$dIix!) contained characters which broke the authentication in multiple ways (# has meaning in URLs, $ has meaning in shell scripts). I also had an exclamation mark, which however seemed fine once I exported the env variable using single instead of double quotes.

Changing the password to an alphanumeric one fixed the issue, as the script proceeded further.

Certificate upload
Here's the (edited) output of ./acme.sh -d "*.subdomain.domain.net" --deploy --deploy-hook synology_dsm --debug 3 --home $PWD.

In short, uploading fails (after quite some time) and the response has an error code of 119. I'd welcome any suggestions here... 🤔

(EDIT: trying to upload the same files through the DSM Web UI works perfectly)

DSM: 6.2.2-24922 Update 4
Wget: GNU Wget 1.19.2

[Tue Apr 28 17:37:41 IST 2020] Upload certificate to the Synology DSM
[Tue Apr 28 17:37:41 IST 2020] ./acme.sh:_post:1733                     POST
[Tue Apr 28 17:37:41 IST 2020] ./acme.sh:_post:1734                     _post_url='http://localhost:5000/webapi/entry.cgi?api=SYNO.Core.Certificate&method=import&version=1&SynoToken=MYSYNOTOKEN'
[Tue Apr 28 17:37:41 IST 2020] ./acme.sh:_post:1735                     body='----------------------------20200428163741
Content-Disposition: form-data; name="key"; filename="*.subdomain.domain.net.key"
Content-Type: application/octet-stream

-----BEGIN RSA PRIVATE KEY-----
<PRIVATE KEY REMOVED>
-----END RSA PRIVATE KEY-----

----------------------------20200428163741
Content-Disposition: form-data; name="cert"; filename="*.subdomain.domain.net.cer"
Content-Type: application/octet-stream

-----BEGIN CERTIFICATE-----
<CERTIFICATE REMOVED>
-----END CERTIFICATE-----

----------------------------20200428163741
Content-Disposition: form-data; name="inter_cert"; filename="ca.cer"
Content-Type: application/octet-stream


-----BEGIN CERTIFICATE-----
<CERTIFICATE REMOVED>
-----END CERTIFICATE-----

----------------------------20200428163741
Content-Disposition: form-data; name="id"


----------------------------20200428163741
Content-Disposition: form-data; name="desc"

Letsencrypt wildcard certificate for my internal network
----------------------------20200428163741
Content-Disposition: form-data; name="as_default"

false
----------------------------20200428163741--
'
[Tue Apr 28 17:37:41 IST 2020] ./acme.sh:_post:1736                     _postContentType='multipart/form-data; boundary=--------------------------20200428163741'
[Tue Apr 28 17:37:42 IST 2020] ./acme.sh:_exists:494                    curl exists=0
[Tue Apr 28 17:37:42 IST 2020] ./acme.sh:_exists:494                    mktemp exists=0
[Tue Apr 28 17:37:42 IST 2020] ./acme.sh:_exists:494                    wget exists=0
[Tue Apr 28 17:37:42 IST 2020] ./acme.sh:_post:1794                     _WGET='wget -q -d  --content-on-error '
[Tue Apr 28 17:40:08 IST 2020] Please refer to https://www.gnu.org/software/wget/manual/html_node/Exit-Status.html for error code: 4
[Tue Apr 28 17:40:08 IST 2020] ./acme.sh:_sed_i:867                     options='s/^ *//g'
[Tue Apr 28 17:40:08 IST 2020] ./acme.sh:_sed_i:869                     Using sed  -i
[Tue Apr 28 17:40:09 IST 2020] ./acme.sh:_post:1843                     _ret='4'
[Tue Apr 28 17:40:09 IST 2020] deploy/synology_dsm.sh:synology_dsm_deploy:142 response='{"error":{"code":119},"success":false}'
[Tue Apr 28 17:40:09 IST 2020] Unable to update certificate, error code {"error":{"code":119},"success":false}
[Tue Apr 28 17:40:09 IST 2020] Error deploy for domain:*.subdomain.domain.net
[Tue Apr 28 17:40:09 IST 2020] Deploy error.

[tatablack]

Actually, according to the manual, Wget does not currently support multipart/form-data for transmitting POST data... 😖

I tried this again using cURL and it works.

So I guess the options are either doing the upload in a way that works with Wget, or bail out with a message if that's not possible and Wget is being used.

[tresni]

Exit code 4 is a network error which I don't think is related to the form data itself. Also, the documentation for wget is just saying that it cannot create multipart uploads itself. We construct the payload ourself and send it as data, I don't think this has any impact on wget, but I'll double check.

[JoJota]

I'm using default cURL for transferring the certificates, but receiving error code 119 too.

As the default synology certificate is not verified, I first got cURL error 60 with the combination of "SSL certificate problem: unable to get local issuer certificate". After resolving this problem with setting export HTTPS_INSECURE=1 I received the synology error 119, which I am stuck with now. Further investigation seemed that there is not very much known what causes this error in detail, at least I couldn't find anything helpful.

Apart from that after not getting any further, I tried importing the certificates with a different reload script which was linked here, but this broke my DSM certificate system completely (no certificates were shown and even after regenerating the nginx certificate folder all the services which use the certificates are gone and I am not able to restore them). So I will now restore the DSM system to factory settings to see if that somehow helps getting rid of the 119er.

Device: DS216j
DSM Version: DSM 6.2.2-24922 Update 4

Error 119 debug log using cURL (cleared of private data):

[Mon  4 May 10:37:42 CEST 2020] _postContentType='multipart/form-data; boundary=--------------------------20200504083742'
[Mon  4 May 10:37:42 CEST 2020] _CURL='curl -L --silent --dump-header /home/pi/.acme.sh/http.header  --trace-ascii /tmp/tmp.DOAhb8GRzk  -g  --insecure  '
[Mon  4 May 10:37:42 CEST 2020] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 92
[Mon  4 May 10:37:42 CEST 2020] Here is the curl dump log:
[Mon  4 May 10:37:42 CEST 2020] == Info: Expire in 0 ms for 6 (transfer 0x3fa880)
== Info: Expire in 1 ms for 1 (transfer 0x3fa880)
== Info: Expire in 0 ms for 1 (transfer 0x3fa880)
(...)
== Info: Expire in 0 ms for 1 (transfer 0x3fa880)
== Info:   Trying 192.168.222.21...
== Info: TCP_NODELAY set
== Info: Expire in 200 ms for 4 (transfer 0x3fa880)
== Info: Connected to example.com (192.168.222.21) port 443 (#0)
== Info: ALPN, offering h2
== Info: ALPN, offering http/1.1
== Info: successfully set certificate verify locations:
== Info:   CAfile: none
  CApath: /etc/ssl/certs
=> Send SSL data, 5 bytes (0x5)
0000: .....
== Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
=> Send SSL data, 512 bytes (0x200)
0000: ......x..*79..%.).$.J.G..}...<..&..r.# ......b..j.G>....};...-
0040: O.,.....>.......,.0.........+./...$.(.k.#.'.g.....9.....3.....=.
<= Recv SSL data, 5 bytes (0x5)
0000: ....b
== Info: TLSv1.3 (IN), TLS handshake, Server hello (2):
<= Recv SSL data, 98 bytes (0x62)
0000: ...^../*.-.....Y.J.......C3]n~........ ..1.!.$..Nz.M.}d...lH...<
0040: .pY}..../.......................h2
<= Recv SSL data, 5 bytes (0x5)
0000: ....(
== Info: TLSv1.2 (IN), TLS handshake, Certificate (11):
<= Recv SSL data, 808 bytes (0x328)
0000: ...$..!...0...0.......|..{My.0...*.H........0Q1.0...U....TW1.0..
0040: .U....Taipei1.0...U....Synology Inc.1.0...U....Synology Inc. CA0
0080: ...200503181130Z..400119181130Z0M1.0...U....TW1.0...U....Taipei1
00c0: .0...U....Synology Inc.1.0...U....synology.com0.."0...*.H.......
0100: ......0..............K.u.v..^Ht.#..:.4......4.w..;..B.d.......
0140: (.'....W.>..(.........qlD.3.'B....EO3..Y.CvT.....X)i%?GE.DE..c
0180: (.]#........RbSn.hL~....1.A.....;..#....k..F...1....]w.G0.....^.
01c0: .......4X.....*..\.^..b_....5.K..u<s...D.-.....FU....._...V..(..
0200: W=4y.......m.-#.....0...*.H..............^.........e....E..O.MiB
0240: /n.....eC!...p..v.C7G...*n...."..N.|....E.).....s..C..t6.bx;..!.
0280: .(.iy!.*...~_p},{.<.R.........Lm.=mu..Q......U.%+H.)u6...T......
02c0: ..e...........9!..6..2.c.V..o.x..-iN2:.e...`....|.*+k.b..4.nc+..
0300: =........t.....h.....X..`xD.....q....*D6
<= Recv SSL data, 5 bytes (0x5)
0000: ....M
== Info: TLSv1.2 (IN), TLS handshake, Server key exchange (12):
<= Recv SSL data, 333 bytes (0x14d)
0000: ...I...A...1...E.sNL..,8.5.....\l.ui.3. q.F..9E.3y1C.....}..3...
0040: ....... .......N$.?./.y5.......> ....e..b..}Cc....:..3.....@....
0080: ..%D.]...{.....V.......H!...#M". ...y.4.<..|.2.i^...;'J..UW..M
00c0: y......4..1..)T.dS.&...=....I..c@W....sM....N.|.$U$.B'.M........
0100: .K..@.......z.|6G...d~..Un......=.....9..Fk........W...s.r{...W.
0140: P..O.u2....MD
<= Recv SSL data, 5 bytes (0x5)
0000: .....
== Info: TLSv1.2 (IN), TLS handshake, Server finished (14):
<= Recv SSL data, 4 bytes (0x4)
0000: ....
=> Send SSL data, 5 bytes (0x5)
0000: ....F
== Info: TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
=> Send SSL data, 70 bytes (0x46)
0000: ...BA..*~b.#.}>i.....?..G..Y.*....l.3..'..T...l6.h.&../..@.I...@
0040: {}.=.E
=> Send SSL data, 5 bytes (0x5)
0000: .....
== Info: TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
=> Send SSL data, 1 bytes (0x1)
0000: .
=> Send SSL data, 5 bytes (0x5)
0000: ....(
== Info: TLSv1.2 (OUT), TLS handshake, Finished (20):
=> Send SSL data, 16 bytes (0x10)
0000: ........=.b....w
<= Recv SSL data, 5 bytes (0x5)
0000: .....
<= Recv SSL data, 5 bytes (0x5)
0000: ....(
== Info: TLSv1.2 (IN), TLS handshake, Finished (20):
<= Recv SSL data, 16 bytes (0x10)
0000: .....%..S.......
== Info: SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
== Info: ALPN, server accepted to use h2
== Info: Server certificate:
== Info:  subject: C=TW; L=Taipei; O=Synology Inc.; CN=synology.com
== Info:  start date: May  3 18:11:30 2020 GMT
== Info:  expire date: Jan 19 18:11:30 2040 GMT
== Info:  issuer: C=TW; L=Taipei; O=Synology Inc.; CN=Synology Inc. CA
== Info:  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
== Info: Using HTTP2, server supports multi-use
== Info: Connection state changed (HTTP/2 confirmed)
== Info: Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
=> Send SSL data, 5 bytes (0x5)
0000: ....0
=> Send SSL data, 5 bytes (0x5)
0000: ....3
=> Send SSL data, 5 bytes (0x5)
0000: ....%
== Info: Using Stream ID: 1 (easy handle 0x3fa880)
=> Send SSL data, 5 bytes (0x5)
0000: .....
=> Send header, 375 bytes (0x177)
0000: POST /webapi/entry.cgi?api=SYNO.Core.Certificate&method=import&v
0040: ersion=1&SynoToken=ObIUVFgjj3w66 HTTP/2
0069: Host: example.com:port
008b: User-Agent: acme.sh/2.8.6 (https://github.com/acmesh-official/ac
00cb: me.sh)
00d3: Accept: */*
00e0: Content-Type: multipart/form-data; boundary=--------------------
0120: ------20200504083742
0136: Cookie: ;
0142: X-SYNO-TOKEN: ObIUVFgjj3w66
015f: Content-Length: 6185
0175:
<= Recv SSL data, 5 bytes (0x5)
0000: ....@
== Info: Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
=> Send SSL data, 5 bytes (0x5)
0000: ....!
=> Send SSL data, 5 bytes (0x5)
0000: ....J
=> Send data, 6185 bytes (0x1829)
0000: ----------------------------20200504083742
002c: Content-Disposition: form-data; name="key"; filename="my.key"
007e: Content-Type: application/octet-stream
00a6:
00a8: -----BEGIN RSA PRIVATE KEY----------END RSA PRIVAT
0728: E KEY-----.
0735: ----------------------------20200504083742
0761: Content-Disposition: form-data; name="cert"; filename="my.cer"
07b4: Content-Type: application/octet-stream
07dc:
07de: -----BEGIN CERTIFICATE----------END C
0f9e: ERTIFICATE-----.
0fb0: ----------------------------20200504083742
0fdc: Content-Disposition: form-data; name="inter_cert"; filename="ca.
101c: cer"
1022: Content-Type: application/octet-stream
104a:
104c: .-----BEGIN CERTIFICATE----------END CERTIFICATE-----.
16be: ----------------------------20200504083742
16ea: Content-Disposition: form-data; name="id"
1715:
1717:
1719: ----------------------------20200504083742
1745: Content-Disposition: form-data; name="desc"
1772:
1774: acme.sh certificate
1793: ----------------------------20200504083742
17bf: Content-Disposition: form-data; name="as_default"
17f2:
17f4: false
17fb: ----------------------------20200504083742--
== Info: We are completely uploaded and fine
<= Recv SSL data, 5 bytes (0x5)
0000: ....!
<= Recv SSL data, 5 bytes (0x5)
0000: ....O
<= Recv header, 13 bytes (0xd)
0000: HTTP/2 200
<= Recv header, 15 bytes (0xf)
0000: server: nginx
<= Recv header, 37 bytes (0x25)
0000: date: Mon, 04 May 2020 08:37:42 GMT
<= Recv header, 43 bytes (0x2b)
0000: content-type: text/plain; charset="UTF-8"
<= Recv header, 23 bytes (0x17)
0000: vary: Accept-Encoding
<= Recv header, 25 bytes (0x19)
0000: x-request-error: unauth
<= Recv header, 33 bytes (0x21)
0000: x-content-type-options: nosniff
<= Recv header, 33 bytes (0x21)
0000: x-xss-protection: 1; mode=block
<= Recv header, 63 bytes (0x3f)
0000: cache-control: max-age=0, no-cache, no-store, must-revalidate
<= Recv header, 18 bytes (0x12)
0000: pragma: no-cache
<= Recv header, 12 bytes (0xc)
0000: expires: 0
<= Recv header, 73 bytes (0x49)
0000: strict-transport-security: max-age=15768000; includeSubdomains;
0040: preload
<= Recv header, 2 bytes (0x2)
0000:
<= Recv data, 38 bytes (0x26)
0000: {"error":{"code":119},"success":false}
<= Recv SSL data, 5 bytes (0x5)
0000: ....%
== Info: HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)
== Info: stopped the pause stream!
== Info: Connection #0 to host example.com left intact
[Mon  4 May 10:37:42 CEST 2020] _ret='92'
[Mon  4 May 10:37:42 CEST 2020] Unable to update certificate, error code {"error":{"code":119},"success":false}
[Mon  4 May 10:37:43 CEST 2020] Error deploy for domain:example.com
[Mon  4 May 10:37:43 CEST 2020] Deploy error.

[tresni]

I've fixed the URL encoding issue for passwords, I'll open a PR for that shortly. I have confirmed there appears to be an issue using wget. Comparing differences between wget and curl now.