Skip to content

[BUG] 400 Bad Request with latest version of shlink docker container #379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
1 task done
gabriel-lando opened this issue Jun 1, 2023 · 5 comments
Closed
1 task done

[BUG] 400 Bad Request with latest version of shlink docker container #379

gabriel-lando opened this issue Jun 1, 2023 · 5 comments

Comments

@gabriel-lando
Copy link

gabriel-lando commented Jun 1, 2023 •
edited
Loading

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

After updating shlink to latest version, which changed the default runtime to Roadrunner, it stopped working when trying to access via swag + nginx. If I try to access it directly (using http://container-ip:8080/short-url it works.

I started a thread at the shlink GH, but what we found is that it seems to be an issue with some swag/nginx configuration (shlinkio/shlink#1796).

I tried to enable "debug" logs on the nginx.conf file, but nothing is being logged.
If I try to deploy a default nginx container, pointing to shlink, it works.

Obs: if use an older tag of the shlink docker image or the current version, but with the openswoole runtime (like shlinkio/shlink:stable-openswoole, it works. The latest version (3.6.0) changes the runtime to Roadrunner, what seems to be the cause of the issue with swag.

Expected Behavior

No response

Steps To Reproduce

  1. In a VM (tested on Ubuntu Server 22.04 and Alpine Linux 3.16)
  2. Create the docker-compose.yaml file (content in the Docker creation section).
  3. Create the .env file (content in the Docker creation section).
  4. Start the docker-compose to create the swag data directory.
  5. Stop the docker-compose.
  6. Add the domain validation settings (in my sample, I'm using the Cloudflare API)
  7. For shlink, add the configuration file in /data/swag/nginx/proxy-confs/url.subdomain.conf (content in the Docker creation section).
  8. Restart the docker-compose to run everything.
  9. To test, try to access https://url.example.com, it should redirect to https://app.shlink.io/ if working, but instead it is showing a 400 Bad Request message. But if you access http://shlink-vm-ip:8080, it redirects you to the right URL.

Environment

  • OS: Ubuntu Server 22.04 up-to-date
  • How docker service was installed: Following this documentation.

CPU architecture

x86-64

Docker creation

  • docker-compose.yaml:
version: "3"

services:
  shlink:
    image: shlinkio/shlink:stable
    pull_policy: always
    container_name: shlink
    restart: unless-stopped
    tty: true
    stdin_open: true
    depends_on:
      - mysql
    external_links:
      - mysql:mysql
    environment:
      IS_HTTPS_ENABLED: "true"
      DEFAULT_DOMAIN: ${SHLINK_DOMAIN}
      DB_DRIVER: ${SHLINK_DB_DRIVER}
      DB_NAME: ${SHLINK_DB_NAME}
      DB_USER: ${SHLINK_DB_USER}
      DB_PASSWORD: ${SHLINK_DB_PASSWORD}
      DB_HOST: ${SHLINK_DB_HOST}
      GEOLITE_LICENSE_KEY: ${SHLINK_GEOLITE_KEY}
      DEFAULT_INVALID_SHORT_URL_REDIRECT: ${SHLINK_INVALID_REDIRECT}
      DEFAULT_REGULAR_404_REDIRECT: ${SHLINK_404_REDIRECT}
      DEFAULT_BASE_URL_REDIRECT: ${SHLINK_URL_REDIRECT}
    networks:
      - swag-network

  swag:
    image: "ghcr.io/linuxserver/swag:latest"
    pull_policy: always
    container_name: swag
    restart: unless-stopped
    tty: true
    stdin_open: true
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ${SWAG_PATH}:/config
    environment:
      PUID: 1000
      PGID: 1000
      TZ: "America/Sao_Paulo"
      URL: ${SWAG_DOMAIN}
      EXTRA_DOMAINS: ${SWAG_EXTRA_DOMAINS}
      VALIDATION: "dns"
      DNSPLUGIN: "cloudflare"
      EMAIL: ${SWAG_EMAIL}
      STAGING: "false"
    cap_add:
      - NET_ADMIN
    networks:
      - swag-network

  mysql:
    image: mysql:latest
    pull_policy: always
    container_name: mysql
    restart: unless-stopped
    tty: true
    stdin_open: true
    environment:
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_ROOT_PASSWORD: ${MYSQL_PASSWORD}
    volumes:
      - ${MYSQL_PATH}:/var/lib/mysql
    networks:
      - swag-network

networks:
  swag-network:
    name: swag
  • .env file (replacing example.com by a valid domain)
# Swag variables
SWAG_PATH="/data/swag"
SWAG_DOMAIN="example.com"
SWAG_EXTRA_DOMAINS="*.example.com"
SWAG_EMAIL="[email protected]"

# MySQL variables
MYSQL_PATH="/data/mysql"
MYSQL_DATABASE="shlink"
MYSQL_PASSWORD="example-password"

# Shlink variables
SHLINK_DOMAIN="example.com"
SHLINK_DB_DRIVER="mysql"
SHLINK_DB_NAME="shlink"
SHLINK_DB_USER="root"
SHLINK_DB_PASSWORD="example-password"
SHLINK_DB_HOST="mysql"
SHLINK_GEOLITE_KEY="example-key"
SHLINK_INVALID_REDIRECT="https://app.shlink.io/"
SHLINK_404_REDIRECT="https://app.shlink.io/"
SHLINK_URL_REDIRECT="https://app.shlink.io/"
  • nginx configuration file (url.subdomain.conf)
## Version 2021/05/18
# make sure that your dns has a cname set for <container_name> and that your <container_name> container is not using a base url

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name url.*;

    set $upstream_app shlink;
    set $upstream_port 8080;
    set $upstream_proto http;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Container logs

  • Docker SWAG Logs:
[migrations] started
[migrations] 01-nginx-site-confs-default: skipped
[migrations] done
───────────────────────────────────────

      ██╗     ███████╗██╗ ██████╗
      ██║     ██╔════╝██║██╔═══██╗
      ██║     ███████╗██║██║   ██║
      ██║     ╚════██║██║██║   ██║
      ███████╗███████║██║╚██████╔╝
      ╚══════╝╚══════╝╚═╝ ╚═════╝

   Brought to you by linuxserver.io
───────────────────────────────────────

To support the app dev(s) visit:
Certbot: https://supporters.eff.org/donate/support-work-on-certbot

To support LSIO projects visit:
https://www.linuxserver.io/donate/

───────────────────────────────────────
GID/UID
───────────────────────────────────────

User UID:    1000
User GID:    1000
───────────────────────────────────────

using keys found in /config/keys
Variables set:
PUID=1000
PGID=1000
TZ=America/Sao_Paulo
URL=example.com
SUBDOMAINS=
EXTRA_DOMAINS=*.example.com
ONLY_SUBDOMAINS=false
VALIDATION=dns
CERTPROVIDER=
DNSPLUGIN=cloudflare
[email protected]
STAGING=false

cp: not replacing '/config/dns-conf/acmedns-registration.json'
cp: not replacing '/config/dns-conf/acmedns.ini'
cp: not replacing '/config/dns-conf/aliyun.ini'
cp: not replacing '/config/dns-conf/azure.ini'
cp: not replacing '/config/dns-conf/cloudflare.ini'
cp: not replacing '/config/dns-conf/cpanel.ini'
cp: not replacing '/config/dns-conf/desec.ini'
cp: not replacing '/config/dns-conf/digitalocean.ini'
cp: not replacing '/config/dns-conf/directadmin.ini'
cp: not replacing '/config/dns-conf/dnsimple.ini'
cp: not replacing '/config/dns-conf/dnsmadeeasy.ini'
cp: not replacing '/config/dns-conf/dnspod.ini'
cp: not replacing '/config/dns-conf/do.ini'
cp: not replacing '/config/dns-conf/domeneshop.ini'
cp: not replacing '/config/dns-conf/duckdns.ini'
cp: not replacing '/config/dns-conf/dynu.ini'
cp: not replacing '/config/dns-conf/gandi.ini'
cp: not replacing '/config/dns-conf/gehirn.ini'
cp: not replacing '/config/dns-conf/godaddy.ini'
cp: not replacing '/config/dns-conf/google-domains.ini'
cp: not replacing '/config/dns-conf/google.json'
cp: not replacing '/config/dns-conf/he.ini'
cp: not replacing '/config/dns-conf/hetzner.ini'
cp: not replacing '/config/dns-conf/infomaniak.ini'
cp: not replacing '/config/dns-conf/inwx.ini'
cp: not replacing '/config/dns-conf/ionos.ini'
cp: not replacing '/config/dns-conf/linode.ini'
cp: not replacing '/config/dns-conf/loopia.ini'
cp: not replacing '/config/dns-conf/luadns.ini'
cp: not replacing '/config/dns-conf/netcup.ini'
cp: not replacing '/config/dns-conf/njalla.ini'
cp: not replacing '/config/dns-conf/nsone.ini'
cp: not replacing '/config/dns-conf/ovh.ini'
cp: not replacing '/config/dns-conf/porkbun.ini'
cp: not replacing '/config/dns-conf/rfc2136.ini'
cp: not replacing '/config/dns-conf/route53.ini'
cp: not replacing '/config/dns-conf/sakuracloud.ini'
cp: not replacing '/config/dns-conf/standalone.ini'
cp: not replacing '/config/dns-conf/transip.ini'
cp: not replacing '/config/dns-conf/vultr.ini'
cp: not replacing '/config/etc/letsencrypt/renewal-hooks/deploy/10-default'
cp: not replacing '/config/etc/letsencrypt/renewal-hooks/post/10-nginx'
cp: not replacing '/config/etc/letsencrypt/renewal-hooks/pre/10-nginx'
Using Let's Encrypt as the cert provider
No subdomains defined
EXTRA_DOMAINS entered, processing
Extra domains processed are:  -d *.example.com
E-mail address entered: [email protected]
dns validation via cloudflare plugin is selected
Certificate exists; parameters unchanged; starting nginx
The cert does not expire within the next day. Letting the cron script handle the renewal attempts overnight (2:08am).
[custom-init] No custom files found, skipping...
crond[365]: crond (busybox 1.36.0) started, log level 5
[ls.io-init] done.
[01-Jun-2023 12:17:23] NOTICE: fpm is running, pid 364
[01-Jun-2023 12:17:23] NOTICE: ready to handle connections
Server ready
  • Nginx Access logs:
10.0.0.51 - - [01/Jun/2023:12:17:33 -0300] "GET / HTTP/2.0" 400 15 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.57"
10.0.0.51 - - [01/Jun/2023:12:17:33 -0300] "GET /favicon.ico HTTP/2.0" 400 15 "https://url.example.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.57"
  • Nginx Error logs (logging level: debug):
2023/06/01 12:17:23 [notice] 361#361: using the "epoll" event method
2023/06/01 12:17:23 [notice] 361#361: nginx/1.24.0
2023/06/01 12:17:23 [notice] 361#361: OS: Linux 5.15.0-72-generic
2023/06/01 12:17:23 [notice] 361#361: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2023/06/01 12:17:23 [notice] 361#361: start worker processes
2023/06/01 12:17:23 [notice] 361#361: start worker process 392
2023/06/01 12:17:23 [notice] 361#361: start worker process 393
2023/06/01 12:17:23 [notice] 361#361: start cache manager process 394
2023/06/01 12:17:23 [notice] 361#361: start cache loader process 395
2023/06/01 12:18:23 [notice] 395#395: http file cache: /var/lib/nginx/cache 0.000M, bsize: 4096
2023/06/01 12:18:23 [notice] 361#361: signal 17 (SIGCHLD) received from 395
2023/06/01 12:18:23 [notice] 361#361: cache loader process 395 exited with code 0
2023/06/01 12:18:23 [notice] 361#361: signal 29 (SIGIO) received
@github-actions
Copy link

github-actions bot commented Jun 1, 2023

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

@flurmind
Copy link

flurmind commented Jun 8, 2023 •
edited
Loading

I have the same problem on swag without services (standalone container):
docker-compose:

---
version: "2.1"
services:
  swag:
    image: lscr.io/linuxserver/swag:latest
    container_name: swag
    cap_add:
      - NET_ADMIN
    environment:
      - PUID=1000
      - PGID=911
      - URL=xxx
      - SUBDOMAINS=www,ph
      - VALIDATION=http
      - CERTPROVIDER=letsencrypt #optional
      - DOCKER_MODS=linuxserver/mods:swag-dashboard #|linuxserver/mods:swag-dbip
    volumes:
      - /srv/ssd/appdata/swag:/config
    ports:
      - 444:443
      - 81:80 #optional
     restart: unless-stopped
networks:
  default:
    external: true
    name: proxy

Log:

      ██╗     ███████╗██╗ ██████╗ 
      ██║     ██╔════╝██║██╔═══██╗
      ██║     ███████╗██║██║   ██║
      ██║     ╚════██║██║██║   ██║
      ███████╗███████║██║╚██████╔╝
      ╚══════╝╚══════╝╚═╝ ╚═════╝ 
   Brought to you by linuxserver.io
───────────────────────────────────────
To support the app dev(s) visit:
Certbot: https://supporters.eff.org/donate/support-work-on-certbot
To support LSIO projects visit:
https://www.linuxserver.io/donate/
───────────────────────────────────────
GID/UID
───────────────────────────────────────
User UID:    1000
User GID:    911
───────────────────────────────────────
using keys found in /config/keys
Variables set:
�
0
PGID=911
�
URL=xxx.com
SUBDOMAINS=www,ph
EXTRA_DOMAINS=
ONLY_SUBDOMAINS=false
VALIDATION=http
CERTPROVIDER=letsencrypt
DNSPLUGIN=
EMAIL=xxx
STAGING=
cp: not replacing '/config/dns-conf/acmedns-registration.json'
cp: not replacing '/config/dns-conf/acmedns.ini'
cp: not replacing '/config/dns-conf/aliyun.ini'
cp: not replacing '/config/dns-conf/azure.ini'
cp: not replacing '/config/dns-conf/cloudflare.ini'
cp: not replacing '/config/dns-conf/cpanel.ini'
cp: not replacing '/config/dns-conf/desec.ini'
cp: not replacing '/config/dns-conf/digitalocean.ini'
cp: not replacing '/config/dns-conf/directadmin.ini'
cp: not replacing '/config/dns-conf/dnsimple.ini'
cp: not replacing '/config/dns-conf/dnsmadeeasy.ini'
cp: not replacing '/config/dns-conf/dnspod.ini'
cp: not replacing '/config/dns-conf/do.ini'
cp: not replacing '/config/dns-conf/domeneshop.ini'
cp: not replacing '/config/dns-conf/duckdns.ini'
cp: not replacing '/config/dns-conf/dynu.ini'
cp: not replacing '/config/dns-conf/gandi.ini'
cp: not replacing '/config/dns-conf/gehirn.ini'
cp: not replacing '/config/dns-conf/godaddy.ini'
cp: not replacing '/config/dns-conf/google-domains.ini'
cp: not replacing '/config/dns-conf/google.json'
cp: not replacing '/config/dns-conf/he.ini'
cp: not replacing '/config/dns-conf/hetzner.ini'
cp: not replacing '/config/dns-conf/infomaniak.ini'
cp: not replacing '/config/dns-conf/inwx.ini'
cp: not replacing '/config/dns-conf/ionos.ini'
cp: not replacing '/config/dns-conf/linode.ini'
cp: not replacing '/config/dns-conf/loopia.ini'
cp: not replacing '/config/dns-conf/luadns.ini'
cp: not replacing '/config/dns-conf/netcup.ini'
cp: not replacing '/config/dns-conf/njalla.ini'
cp: not replacing '/config/dns-conf/nsone.ini'
cp: not replacing '/config/dns-conf/ovh.ini'
cp: not replacing '/config/dns-conf/porkbun.ini'
cp: not replacing '/config/dns-conf/rfc2136.ini'
cp: not replacing '/config/dns-conf/route53.ini'
cp: not replacing '/config/dns-conf/sakuracloud.ini'
cp: not replacing '/config/dns-conf/standalone.ini'
cp: not replacing '/config/dns-conf/transip.ini'
cp: not replacing '/config/dns-conf/vultr.ini'
cp: not replacing '/config/etc/letsencrypt/renewal-hooks/deploy/10-default'
cp: not replacing '/config/etc/letsencrypt/renewal-hooks/post/10-nginx'
cp: not replacing '/config/etc/letsencrypt/renewal-hooks/pre/10-nginx'
Using Let's Encrypt as the cert provider
SUBDOMAINS entered, processing
SUBDOMAINS entered, processing
Sub-domains processed are:  xxxxxxxxxxxxxxxxxxxxxxxxxx
E-mail address entered: xxxxxxxxxxxxxxxxxxxxx
http validation is selected
Certificate exists; parameters unchanged; starting nginx
The cert does not expire within the next day. Letting the cron script handle the renewal attempts overnight (2:08am).
**** Applying the SWAG dashboard mod... ****
**** goaccess already installed, skipping ****
**** libmaxminddb already installed, skipping ****
**** Applied the SWAG dashboard mod ****
[custom-init] No custom files found, skipping...
[ls.io-init] done.
nginx: [warn] protocol options redefined for 0.0.0.0:443 in /config/nginx/proxy-confs/nextcloud.subdomain.conf:18
nginx: [warn] protocol options redefined for [::]:443 in /config/nginx/proxy-confs/nextcloud.subdomain.conf:19
�
Server ready

@nemchik
Copy link
Member

nemchik commented Jun 9, 2023

@flurmind you have two different issues unrelated to the topic of this issue. The @gabriel-lando also has one of the issues.

  • cp: not replacing is an issue we're aware of, but it does not cause anything to stop working, it's just spammy in the logs.
  • nginx: [warn] protocol options redefined for also shouldn't cause anything to stop working, but this one is an easy fix. open the nginx conf for the swag dashboard and change: 
      listen 443 ssl;
  listen [::]:443 ssl;
    to 
      listen 443 ssl http2;
  listen [::]:443 ssl http2;

@gabriel-lando looking at shlinkio/shlink#1796 the dev replied that they had no issue using a vanilla nginx container with a similar config, but one specific line is missing in their config because it's specific to swag:

        include /config/nginx/proxy.conf;

Could you try removing that line from your shlink proxy?

I don't think that should be considered the final fix, but if I'm correct it could lead us to a more correct solution.

@gabriel-lando
Copy link
Author

gabriel-lando commented Jun 9, 2023 •
edited
Loading

Hi @nemchik thanks for your help :)

You are right, I commented out this proxy.conf line and added the http2 and now it worked :)

Update:
I started commenting out line by line of the proxy.conf file until find which one was causing the issue.
I discovered that, after commenting out the line proxy_set_header Host $host; (Line 24) it works. I don't know exactly why yet :D
I restored the proxy.conf file to the original value and the issue happened again, commenting out that single line, the issue disappeared. So now we have a point to investigate.

Update 2:
I noticed that, in my url.subdomain.conf file, I already have the proxy_set_header Host $host; line. So the nginx was setting the same header twice (it should override, right???).
After removing this proxy_set_header from my config file and reseting the proxy.conf to it's original state, it worked again.
I don't know if it's an issue with nginx or swag (or if it's a real issue), but it seems that if I set the Host header twice, I receive a 400 Bad Request from the shlink.

Update 3:
It seems that it's totally my bad :( I created the shlink config file based on the reverse proxy documentation. But it was made for vanilla nginx. So I added a lot of proxy_set_header duplicated from the proxy.conf. I removed everything and now it's working well :)

@nemchik
Copy link
Member

nemchik commented Jun 19, 2023

I would be fine with shipping a config in swag that uses https://github.com/linuxserver/reverse-proxy-confs/blob/master/_template.subdomain.conf.sample as the template (basically just follow the instructions in the file comments). I don't think any of the extra proxy_* lines you have in your config are needed since we ship all of them by default except the real ip line. For the real IP I would recommend our docker mod https://github.com/linuxserver/docker-mods/tree/swag-cloudflare-real-ip if you're using cloudflare. If you're not using cloudflare, you can still use the mod and it will accomplish the same thing, or you can set proxy_set_header X-Real-IP $remote_addr; the way you have.

@nemchik nemchik closed this as completed Jun 19, 2023
@LinuxServer-CI LinuxServer-CI moved this from Issues to Done in Issue & PR Tracker Jun 19, 2023
@gabriel-lando gabriel-lando mentioned this issue Jun 19, 2023
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
Issue & PR Tracker
Archived in project
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nemchik @gabriel-lando @flurmind