Skip to content

youtube-dl and ffprobe requires libblas.so #3506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
3 tasks done
hyiltiz opened this issue Jul 9, 2020 · 56 comments
Open
3 tasks done

youtube-dl and ffprobe requires libblas.so #3506

hyiltiz opened this issue Jul 9, 2020 · 56 comments

Comments

@hyiltiz
Copy link

hyiltiz commented Jul 9, 2020

Write clear, concise and in textual form.

Bug and expected behavior

  • youtube-dl -x SOMELINK fails.
  • youtube-dl -x SOMELINK succeeds.

No profile or disabling firejail

  • firejail --noprofile ffprobe SOMEFILE succeeds.
  • /usr/bin/ffprobe SOMEFILE succeeds.

Reproduce
Steps to reproduce the behavior:

  1. Run in bash firejail youtube-dl -x SOMELINK or firejail ffprobe SOMEFILE
  2. See error:
> firejail ffprobe SOMEFILE.webm
Error fcopy: size limit of 500 MB reached
Error getpwuid: main.c:294 init_cfg: No such file or directory

> ffprobe SOMEFILE.webm
Error fcopy: size limit of 500 MB reached
/usr/bin/ffprobe: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory

> /usr/bin/ffprobe SOMEFILE.webm
ffprobe version 4.3-2 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Debian 9.3.0-13)
...

Environment

  • Linux distribution and version (ie output of lsb_release -a)
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux bullseye/sid
Release:        testing
Codename:       bullseye
  • Firejail version (output of firejail --version) exclusive or used git commit (git rev-parse HEAD)
firejail version 0.9.62

Compile time support:
        - AppArmor support is enabled
        - AppImage support is enabled
        - chroot support is enabled
        - file and directory whitelisting support is enabled
        - file transfer support is enabled
        - firetunnel support is enabled
        - networking support is enabled
        - overlayfs support is enabled
        - private-home support is enabled
        - seccomp-bpf support is enabled
        - user namespace support is enabled
        - X11 sandboxing support is enabled
  • What other programs interact with the affected program for the functionality?
    youtube-dl depends on ffprobe.
  • Are these listed in the profile?
    Huh?

Checklist

  • The upstream profile (and redirect profile if exists) have no changes fixing it.
  • The upstream profile exists (find / -name 'firejail' 2>/dev/null/fd firejail to locate profiles ie in /usr/local/etc/firejail/PROGRAM.profile)
  • [x Programs needed for interaction are listed.
  • Error was checked in search engine and on issue list without success.
OUTPUT OF `firejail --debug PROGRAM`
https://termbin.com/3iou

BTW, I just noticed that the above firejail --debug provided way too much information about my filesystem and setup. Was that necessary? If so, please consider creating a key pair and uploading your public key so these kinds of probably sensitive private information can be shared directly with the project without putting it up in the Internet indefinitely in plain text.

@bbhtt
Copy link
Contributor

bbhtt commented Jul 10, 2020

Error fcopy: size limit of 500 MB reached

See here

Error getpwuid: main.c:294 init_cfg: No such file or directory

Can you add passwd,groups,login.defs to private-etc and see if it persists or comment it to see? Is there something wrong with the first line? I ran the command,it works fine:

Output
firejail --ignore=quiet youtube-dl --extract-audio --audio-format mp3 https://www.youtube.com/watch?v=vWwgrjjIMXA
Reading profile /etc/firejail/youtube-dl.profile
Reading profile /etc/firejail/allow-python2.inc
Reading profile /etc/firejail/allow-python3.inc
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-devel.inc
Reading profile /etc/firejail/disable-exec.inc
Reading profile /etc/firejail/disable-interpreters.inc
Reading profile /etc/firejail/disable-passwdmgr.inc
Reading profile /etc/firejail/disable-programs.inc
Reading profile /etc/firejail/disable-xdg.inc
Reading profile /etc/firejail/whitelist-usr-share-common.inc
Reading profile /etc/firejail/whitelist-var-common.inc
Warning: networking feature is disabled in Firejail configuration file
Parent pid 8510, child pid 8511
Warning: skipping crypto-policies for private /etc
Warning: skipping youtube-dl.conf for private /etc
Private /etc installed in 42.92 ms
25 programs installed in 79.52 ms
Warning: An abstract unix socket for session D-BUS might still be available. Use --net or remove unix from --protocol set.
Blacklist violations are logged to syslog
Child process initialized in 236.85 ms
[youtube] vWwgrjjIMXA: Downloading webpage
[download] Destination: Blowing In The Wind (Live On TV, March 1963)-vWwgrjjIMXA.webm
[download] 100% of 2.45MiB in 00:03
[ffmpeg] Destination: Blowing In The Wind (Live On TV, March 1963)-vWwgrjjIMXA.mp3
Deleting original file Blowing In The Wind (Live On TV, March 1963)-vWwgrjjIMXA.webm (pass -k to keep)

Parent is shutting down, bye...

/usr/bin/ffprobe: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory

ldconfig -p | grep -i "libblas" shows anything?

@hyiltiz
Copy link
Author

hyiltiz commented Jul 10, 2020

Error fcopy: size limit of 500 MB reached

See here
Seems it is compiled into firejail; not much I can do unless recompile myself... It would be great to be as a config parameter though.

Error getpwuid: main.c:294 init_cfg: No such file or directory

Can you add passwd,groups,login.defs to private-etc and see if it persists or comment it to see? Is there something wrong with the first line? I ran the command,it works fine:

$ firejail --private-etc=passwd,groups,login.defs, ffprobe SOMEFILE.webm
Error fcopy: size limit of 500 MB reached
/usr/bin/ffprobe: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory

Output

/usr/bin/ffprobe: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory

ldconfig -p | grep -i "libblas" shows anything?

$ ldconfig -p | grep -i "libblas"
        libblas.so.3 (libc6,x86-64) => /lib/x86_64-linux-gnu/libblas.so.3
        libblas.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libblas.so

@glitsj16
Copy link
Collaborator

Error fcopy: size limit of 500 MB reached

This can be set in /etc/firejail/firejail.config:

[...]
# Set the limit for file copy in several --private-* options. The size is set
# in megabytes. By default we allow up to 500MB.
# Note: the files are copied in RAM.
# file-copy-limit 500
[...]

Can you post output of

$ firejail --private-etc=ld.so.cache /usr/bin/ffprobe https://www.youtube.com/watch?v=vWwgrjjIMXA

@bbhtt
Copy link
Contributor

bbhtt commented Jul 11, 2020

$ firejail --private-etc=ld.so.cache /usr/bin/ffprobe https://www.youtube.com/watch?v=vWwgrjjIMXA

ffprobe cannot parse direct web links,you need a direct link to the video,not the https site,I think. https://ia800705.us.archive.org/13/items/Jolly_Fish_1932/Tom_and_Jerry_Jolly_Fish_1932_512kb.mp4
I couldn't reproduce this;ffmpeg profiles needed to be changed: ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload from current in git

@hyiltiz
Copy link
Author

hyiltiz commented Jul 11, 2020

hyiltiz ~ $ firejail --private-etc=ld.so.cache /usr/bin/youtube-dl https://www.youtube.com/watch?v=vWwgrjjIMXA
Error fcopy: size limit of 500 MB reached
[youtube] vWwgrjjIMXA: Downloading webpage
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[download] Destination: Blowing In The Wind (Live On TV, March 1963)-vWwgrjjIMXA.f136.mp4
[download] 100% of 7.57MiB in 00:00
[download] Destination: Blowing In The Wind (Live On TV, March 1963)-vWwgrjjIMXA.f251.webm
[download] 100% of 2.45MiB in 00:00
[ffmpeg] Merging formats into "Blowing In The Wind (Live On TV, March 1963)-vWwgrjjIMXA.mkv"
ERROR: ffmpeg: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory
[2]+  Done                    zotero


hyiltiz ~ $ firejail --private-etc=ld.so.cache /usr/bin/ffprobe Blowing\ In\ The\ Wind\ \(Live\ On\ TV\,\ March\ 1963\)-vWwgrjjIMXA.f251.webm
Error fcopy: size limit of 500 MB reached
/usr/bin/ffprobe: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory

hyiltiz ~ $ firejail --private-etc=ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload /usr/bin/ffprobe Blowing\ In\ The\ Wind\ \(Live\ On\ TV\,\ March\ 1963\)-vWwgrjjIMXA.f251.webm
Error fcopy: size limit of 500 MB reached
/usr/bin/ffprobe: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory

@rusty-snake
Copy link
Collaborator

@hyiltiz has you made any modifications to the ffprobe, ffmpeg, youtube-dl profiles?

Error getpwuid: main.c:294 init_cfg: No such file or directory

@kortewegdevries that happens if you do firejail APP where APP has a symlink from firecfg.

@hyiltiz
Copy link
Author

hyiltiz commented Jul 12, 2020

@rusty-snake I don't recall any modifications. My profiles:

$ cat /etc/firejail/ffmpeg.profile /etc/firejail/ffprobe.profile /etc/firejail/youtube-dl.profile | nc termbin.com 9999
https://termbin.com/eff2

@rusty-snake
Copy link
Collaborator

Looks then like you need to try which line is the issue. Copy the ffmpeg profile to your home and comment all line. Then uncomment line for line. Use firejail --profile=ffmpeg.profile ffprobe … to start. Note1: private-bin should commented all the time.
Note2: My feeling is private-etc but I'm not sure.

@hyiltiz
Copy link
Author

hyiltiz commented Jul 12, 2020

Tested with firejail --profile=ffmpeg.profile ffprobe ~/Downloads/media/SOMEFILE.webm after copying /etc/firejail/ffmpeg.profile to current directory ~/.config/firejail.

Changing nothing:

> firejail --profile=ffmpeg.profile ffprobe ~/Downloads/media/SOMEFILE.webm
Error fcopy: size limit of 500 MB reached
Error: no suitable ffprobe executable found

Commenting out private-bin:

> firejail --profile=ffmpeg.profile ffprobe ~/Downloads/media/SOMEFILE.webm
Error fcopy: size limit of 500 MB reached
Error getpwuid: main.c:294 init_cfg: No such file or directory

Also commenting out private-etc worked!

> firejail --profile=ffmpeg.profile ffprobe ~/Downloads/media/SOMEFILE.webm
ffprobe version 4.3-3 Copyright (c) 2007-2020 the FFmpeg developers
...
    encoder         : google/video-file
  Duration: 00:41:43.18, start: -0.007000, bitrate: 138 kb/s
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)

Any idea why private-bin and private-etc are uncommented in master?

@rusty-snake
Copy link
Collaborator

private-bin: the ffprobe.profile contains ignore private-bin
private-etc: try to use the private-etc from master, it has some fixes for dynamic linked librarys.

@hyiltiz
Copy link
Author

hyiltiz commented Jul 13, 2020

Can I just put the master private-etc under ~/.config/firejail/?

@rusty-snake
Copy link
Collaborator

Yes you can add private-etc ... to .config/firejail/ffmpeg.local. Alternative you can also edit /etc/firejail/ffmpeg.profile since the next firejail release will fix it anyway.

ytdl needs ffprobe added to its private-bin (also on master).

@hyiltiz
Copy link
Author

hyiltiz commented Jul 13, 2020

Got the private-etc ... line from master

private-etc alternatives,ca-certificates,crypto-policies,hosts,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,nsswitch.conf,pkcs11,pki,resolv.conf,ssl

to /etc/firejail/ffmpeg.profile, and added ffprobe to /etc/firejail/youtube-dl.profile's private-bin line before youtube-dl like:

private-bin env,ffmpeg,ffprobe,python*,youtube-dl

Still getting (other) errors about ffprobe:

> youtube-dl -x 'https://www.youtube.com/watch?v=abcde'
Error fcopy: size limit of 500 MB reached
[youtube] abcde: Downloading webpage
[download] SOMEFILE.webm has already been downloaded
[download] 100% of 41.22MiB
ERROR: WARNING: unable to obtain file audio codec with ffprobe
> ffprobe SOMEFILE.webm
ffprobe version 4.3-3 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Debian 9.3.0-14)
...
Input #0, matroska,webm, from 'SOMEFILE.webm':
  Metadata:
    encoder         : google/video-file
  Duration: 00:41:43.18, start: -0.007000, bitrate: 138 kb/s
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)

@rusty-snake
Copy link
Collaborator

I just saw that the private-etc line in youtube-dl.profile also needs to extended with ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload.

@hyiltiz
Copy link
Author

hyiltiz commented Jul 13, 2020

Added, but still same error ERROR: WARNING: unable to obtain file audio codec with ffprobe

@rusty-snake
Copy link
Collaborator

$ fjp diff ffmpeg youtube-dl
The following options are unique to ffmpeg.profile:
include ffmpeg.local
whitelist /usr/share/devedeng
whitelist /usr/share/ffmpeg
whitelist /usr/share/qtchooser
protocol protocol inet,inet6

seccomp !set_mempolicy
private-bin ffmpeg
private-etc alternatives,ca-certificates,crypto-policies,hosts,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,nsswitch.conf,pkcs11,pki,resolv.conf,ssl

The following options are unique to youtube-dl.profile:
include youtube-dl.local
ignore noexec ${HOME}
noblacklist ${HOME}/.cache/youtube-dl
noblacklist ${HOME}/.config/youtube-dl
noblacklist ${HOME}/.netrc
include allow-python2.inc
include allow-python3.inc
blacklist /tmp/.X11-unix
blacklist ${RUNUSER}/wayland-*
blacklist ${RUNUSER}
no3d
protocol protocol unix,inet,inet6

seccomp
private-bin env,ffmpeg,python*,youtube-dl
private-etc alternatives,ca-certificates,crypto-policies,hostname,hosts,ld.so.cache,mime.types,pki,resolv.conf,ssl,youtube-dl.conf

Lets go trough it. whitelist /usr/share/ffmpeg looks most promising to me right now, try to add it to ytdl.

@hyiltiz
Copy link
Author

hyiltiz commented Jul 13, 2020

Same error; even tried appending the diff'ed lines:

include ffmpeg.local
whitelist /usr/share/devedeng
whitelist /usr/share/ffmpeg
whitelist /usr/share/qtchooser
protocol protocol inet,inet6

seccomp !set_mempolicy
private-bin ffmpeg
private-etc alternatives,ca-certificates,crypto-policies,hosts,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,nsswitch.conf,pkcs11,pki,resolv.conf,ssl

to /etc/firejail/youtube-dl, but got the same error.

@bbhtt
Copy link
Contributor

bbhtt commented Jul 13, 2020

Can you see if --prefer-ffmpeg works? Or attach verbose and ignore-config flags to youtube-dl and post the output in a text file.

@hyiltiz
Copy link
Author

hyiltiz commented Jul 13, 2020

Here you go (these still keep all the changes we've already done earlier):

> youtube-dl --prefer-ffmpeg -x 'https://www.youtube.com/watch?v=abcd'
Error fcopy: size limit of 500 MB reached
Error fcopy: size limit of 500 MB reached
[youtube] SOMEFILE: Downloading webpage
[download] SOMEFILE.webm has already been downloaded
[download] 100% of 41.22MiB
ERROR: WARNING: unable to obtain file audio codec with ffprobe

> youtube-dl --prefer-ffmpeg --verbose --ignore-config -x 'https://www.youtube.com/watch?v=abcd'
Error fcopy: size limit of 500 MB reached
Error fcopy: size limit of 500 MB reached
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--prefer-ffmpeg', '--verbose', '--ignore-config', '-x', 'https://www.youtube.com/watch?v=abcd']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.06.16.1
[debug] Python version 3.8.4rc1 (CPython) - Linux-5.7.0-1-amd64-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg present, ffprobe present
[debug] Proxy map: {}
[youtube] abcd: Downloading webpage
[debug] Invoking downloader on 'https:/SOMELINK'
[download] SOMEFILE.webm has already been downloaded
[download] 100% of 41.22MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:SOMEFILE.webm'
ERROR: WARNING: unable to obtain file audio codec with ffprobe
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 2065, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/lib/python3/dist-packages/youtube_dl/postprocessor/ffmpeg.py", line 274, in run
    raise PostProcessingError('WARNING: unable to obtain file audio codec with ffprobe')
youtube_dl.utils.PostProcessingError: WARNING: unable to obtain file audio codec with ffprobe

@bbhtt
Copy link
Contributor

bbhtt commented Jul 14, 2020

Just a guess, can you try with a shorter video (10-20 MB) or a different file format like mp4,mkv,m4a; also with specifying the format --audio-format? I couldn't reproduce any of it on Debian 10

@hyiltiz
Copy link
Author

hyiltiz commented Jul 14, 2020

Here, a 1.14MiB video, .m4a format, specifying --audio-format opus:

$ youtube-dl -x 'https://www.youtube.com/watch?v=7RMz7tePA98'
Error fcopy: size limit of 500 MB reached
Error fcopy: size limit of 500 MB reached
[youtube] 7RMz7tePA98: Downloading webpage
[youtube] 7RMz7tePA98: Downloading js player 5253ac4d
[youtube] 7RMz7tePA98: Downloading js player 5253ac4d
[youtube] 7RMz7tePA98: Downloading MPD manifest
[dashsegments] Total fragments: 9
[download] Destination: Firejail Introduction-7RMz7tePA98.m4a
[download] 100% of 1.14MiB in 00:02
[ffmpeg] Correcting container in "Firejail Introduction-7RMz7tePA98.m4a"
ERROR: ffmpeg: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory

$ youtube-dl -x --audio-format opus 'https://www.youtube.com/watch?v=7RMz7tePA98'
Error fcopy: size limit of 500 MB reached
Error fcopy: size limit of 500 MB reached
[youtube] 7RMz7tePA98: Downloading webpage
[youtube] 7RMz7tePA98: Downloading js player 5253ac4d
[youtube] 7RMz7tePA98: Downloading js player 5253ac4d
[youtube] 7RMz7tePA98: Downloading MPD manifest
[dashsegments] Total fragments: 9
[download] Destination: Firejail Introduction-7RMz7tePA98.m4a
[download] 100% of 1.14MiB in 00:01
[ffmpeg] Correcting container in "Firejail Introduction-7RMz7tePA98.m4a"
ERROR: ffmpeg: error while loading shared libraries: libblas.so.3: cannot open shared object file: No such file or directory

@bbhtt
Copy link
Contributor

bbhtt commented Aug 3, 2020

@hyiltiz Trying this one more time. Can you verify if you have the following files?

/etc/alternatives/libblas.so.3-x86_64-linux-gnu (This is a symlink to 3)
/usr/lib/x86_64-linux-gnu/libblas.so.3  (This one's done!)
/usr/lib/x86_64-linux-gnu/blas/libblas.so.3 (3)
/usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
/var/lib/dpkg/alternatives/libblas.so.3-x86_64-linux-gnu

ffprobe needs the library: openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libblas.so.3", O_RDONLY|O_CLOEXEC) = 3

@hyiltiz
Copy link
Author

hyiltiz commented Aug 3, 2020

root@iPhone:~# ls -la /etc/alternatives/libblas.so*
lrwxrwxrwx 1 root root 55 Nov  3  2019 /etc/alternatives/libblas.so.3-x86_64-linux-gnu -> /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
lrwxrwxrwx 1 root root 41 Jan  6  2020 /etc/alternatives/libblas.so-x86_64-linux-gnu -> /usr/lib/x86_64-linux-gnu/blas/libblas.so

root@iPhone:~# ls -la /usr/lib/x86_64-linux-gnu/libblas.so*
lrwxrwxrwx 1 root root 45 Jan  6  2020 /usr/lib/x86_64-linux-gnu/libblas.so -> /etc/alternatives/libblas.so-x86_64-linux-gnu
lrwxrwxrwx 1 root root 47 Mar  9  2019 /usr/lib/x86_64-linux-gnu/libblas.so.3 -> /etc/alternatives/libblas.so.3-x86_64-linux-gnu

root@iPhone:~# ls -la /usr/lib/x86_64-linux-gnu/blas/libblas.so.*
lrwxrwxrwx 1 root root     16 Apr  2 05:56 /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 -> libblas.so.3.9.0
-rw-r--r-- 1 root root 440160 Apr  2 05:56 /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0

root@iPhone:~# ls -la /var/lib/dpkg/alternatives/libblas.so*
-rw-r--r-- 1 root root 201 Aug  2 06:43 /var/lib/dpkg/alternatives/libblas.so.3-x86_64-linux-gnu
-rw-r--r-- 1 root root 421 Apr  6 12:44 /var/lib/dpkg/alternatives/libblas.so-x86_64-linux-gnu

@bbhtt
Copy link
Contributor

bbhtt commented Aug 3, 2020

Hm, I was under the impression that the symlink under /etc/alternatives/ wasn't pointing to the correct library in /lib/, because I could reproduce that error by installing multiple packages (amd64) that offered the same library. Each time I installed one update-alternatives reconfigured that symlink under /etc/and ffprobe searches for the library under /etc/alternatives/. A fresh Debian Sid comes with only libblas3 pre-installed. This was the stock symlink that worked:
/etc/alternatives/libblas.so.3-x86_64-linux-gnu -> /usr/lib/x86_64-linux-gnu/blas/libblas.so.3

@hyiltiz
Copy link
Author

hyiltiz commented Aug 4, 2020

I have libblas3 and libblas-dev installed; I get rid of libblas-dev? Not sure what you are suggesting.

@bbhtt
Copy link
Contributor

bbhtt commented Aug 4, 2020

I have libblas3 and libblas-dev installed; I get rid of libblas-dev? Not sure what you are suggesting.

I'm not to sure about it. I guess it's worth a try at this point. You might also have libopenblas0-pthread. So get rid of all 3+ffmpeg (apt-mark hold/unhold), then reinstall only libblas3,ffmpeg,reset the cache sudo ldconfig -N -X -v,try running with the profile that came with 0.9.62 without any modifications first, then add private-etc of master if you need.I don't know what's wrong with you system but I grabbed a copy of Sid NetInst iso (-344 Mb) took 30 minutes to install and setup firejail and I can run fine with 0.9.62-3 and the profiles that came with it w/o any modifications. The libblas dependency came in with Sid, you can apt-rdepends ffmpeg to find it's listed under libalapack and libsphinx.

@hyiltiz
Copy link
Author

hyiltiz commented Aug 4, 2020

Hmm, I'd be willing to try the above suggestion, but I think it is a workaround rather than a fix if I understand Firejail's "no fuss" design principle correctly. It is better implemented as a rule against various system states rather than fine tuning system packages.

libblas-dev or libblas3* cannot be removed as it is a dependency for a few other packages I'd rather keep. Removed libopenblas0-pthread, libopenblas0* and libopenblas0-openmp*. Removed then installed ffmpeg, then:

~ # ldconfig -N -X -v|grep blas
        libf77blas.so.3 -> libf77blas.so.3.10.3
        libgslcblas.so.0 -> libgslcblas.so.0.0.0
        libcblas.so.3 -> libcblas.so.3.10.3

Even with the private-etc modifications, still getting the same error about libblas.

@bbhtt
Copy link
Contributor

bbhtt commented Aug 4, 2020

libblas-dev or libblas3* cannot be removed as it is a dependency for a few other packages I'd rather keep.

apt install --reinstall libblas3

@hyiltiz
Copy link
Author

hyiltiz commented Aug 4, 2020

Reinstalled libblas3 then ffmpeg hten reset ldconfig cache. Still the same error.

@bbhtt
Copy link
Contributor

bbhtt commented Aug 4, 2020

Commenting private-etc still works? Like you said:

Also commenting out private-etc worked!

@hyiltiz
Copy link
Author

hyiltiz commented Aug 4, 2020

Hmm, I re-downloaded the ffmpeg.profile from master. Had to comment out private-bin ffmpeg, private-etc, and ADDITIONALLY also include disable-shell.inc, dbus-user none and dbus-system none for firejail --profile=~/.config/firejail/ffmpeg.profile ffprobe ~/Downloads/media/SOMEFILE.webm to work.

Uncommenting private-etc ... for the above command says:

Error fcopy: size limit of 500 MB reached
Error getpwuid: main.c:294 init_cfg: No such file or directory
hyiltiz@iPhone /tmp> locate libblas.so
/etc/alternatives/libblas.so-x86_64-linux-gnu
/etc/alternatives/libblas.so.3-x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libblas.so
/usr/lib/x86_64-linux-gnu/libblas.so.3
/usr/lib/x86_64-linux-gnu/atlas/libblas.so.3
/usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3
/usr/lib/x86_64-linux-gnu/blas/libblas.so
/usr/lib/x86_64-linux-gnu/blas/libblas.so.3
/usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
/var/lib/dpkg/alternatives/libblas.so-x86_64-linux-gnu
/var/lib/dpkg/alternatives/libblas.so.3-x86_64-linux-gnu

@bbhtt
Copy link
Contributor

bbhtt commented Aug 4, 2020

@kortewegdevries that happens if you do firejail APP where APP has a symlink from firecfg.

firejail --profile=<> /usr/bin/<> . Those were not in 0.9.62.

@hyiltiz
Copy link
Author

hyiltiz commented Aug 15, 2020

firejail --profile=<> /usr/bin/<> . Those were not in 0.9.62.

Not sure about your <> notation.

@bbhtt
Copy link
Contributor

bbhtt commented Aug 16, 2020

Not sure about your <> notation.

That error happens when you already have a symlink. firejail --profile=youtube-dl /usr/bin/youtube-dl.

@hyiltiz
Copy link
Author

hyiltiz commented Aug 16, 2020

Not sure the last command was what you'd like me to test, but here is what I did:

  1. Downloaded the ffmpeg.profile and youtube-dl.profile from master
  2. Commented out private-bin, private-etc, include disable-shell.inc, dbus-user, dbus-system lines
  3. Ran firejail --profile=~/.config/firejail/youtube-dl.profile /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'

So far, it is the only setup that works reliably.

@bbhtt
Copy link
Contributor

bbhtt commented Aug 16, 2020 via email

@hyiltiz
Copy link
Author

hyiltiz commented Aug 16, 2020

  1. The master doesn't work without any change;
  2. dbus-* etc. needs to be removed for 0.9.62 for firejail to parse first so did
  3. only commenting out private-bin fails
  4. only commenting out private-etc succeeds

For detailed procedures, here is what I did (script typescript output, can be viewed as plain-text file)
http://ix.io/2ujM

@bbhtt
Copy link
Contributor

bbhtt commented Aug 16, 2020

only commenting out private-etc succeeds

This is for ffmpeg or youtube-dl or both? Sorry, I can't open your link...

@hyiltiz
Copy link
Author

hyiltiz commented Aug 16, 2020

Both. As you asked, I tested for ffmpeg/ffprobe only first, then with a successful ffmpeg setup, tested youtube-dl.

@bbhtt
Copy link
Contributor

bbhtt commented Aug 17, 2020

Try adding this to the private-etc line of youtube-dl.profile separated by commas, remove if one is duplicate, "ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,httpd,apache,apache2,nsswitch.conf,host.conf,gai.conf" to see if it works:
firejail --profile=~/.config/firejail/youtube-dl.profile /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'. You can remove the quiet from ffprobe,ffmpeg,ytdl profile to get a better overview of which profiles are being loaded by firejail...

If this still fails, try a combination of strace (strace -o file.txt /usr/bin/ytdl <link> and cat file.txt|egrep "/etc/|/usr/share/" or under firejail: firejail --allow-debuggers --profile=profile strace -o file.txt /usr/bin/ytdl <link>) and firejail --build /usr/bin/ytdl( should output info on terminal,if not look in /tmp for a file named firejail_straceXXX`) to see if something comes up specific to your system that isn't in any of the profiles.

@hyiltiz
Copy link
Author

hyiltiz commented Oct 9, 2020

Try adding this to the private-etc line of youtube-dl.profile separated by commas, remove if one is duplicate, "ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,httpd,apache,apache2,nsswitch.conf,host.conf,gai.conf"

Did that.

 $ cat ~/.config/firejail/youtube-dl.profile 
# Firejail profile for youtube-dl
# Description: Downloader of videos from YouTube and other sites
# This file is overwritten after every install/update
quiet
# Persistent local customizations
include youtube-dl.local
# Persistent global definitions
include globals.local

# breaks when installed under ${HOME} via `pip install --user` (see #2833)
ignore noexec ${HOME}

noblacklist ${HOME}/.cache/youtube-dl
noblacklist ${HOME}/.config/youtube-dl
noblacklist ${HOME}/.netrc
noblacklist ${MUSIC}
noblacklist ${VIDEOS}

# Allow python (blacklisted by disable-interpreters.inc)
include allow-python2.inc
include allow-python3.inc

blacklist /tmp/.X11-unix
blacklist ${RUNUSER}/wayland-*
blacklist ${RUNUSER}

include disable-common.inc
include disable-devel.inc
include disable-exec.inc
include disable-interpreters.inc
include disable-passwdmgr.inc
include disable-programs.inc
# include disable-shell.inc
include disable-xdg.inc

include whitelist-usr-share-common.inc
include whitelist-var-common.inc

apparmor
caps.drop all
ipc-namespace
machine-id
netfilter
no3d
nodvd
nogroups
nonewprivs
noroot
nosound
notv
nou2f
novideo
protocol unix,inet,inet6
seccomp
shell none
tracelog

private-bin env,ffmpeg,python*,youtube-dl
private-cache
private-dev
# private-etc alternatives,ca-certificates,crypto-policies,hostname,hosts,ld.so.cache,mime.types,pki,resolv.conf,ssl,youtube-dl.conf
private-etc ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,httpd,apache,apache2,nsswitch.conf,host.conf,gai.conf
private-tmp

# dbus-user none
# dbus-system none

#memory-deny-write-execute - breaks on Arch (see issue #1803)
hyiltiz@iPhone ~ $ cat ~/.config/firejail/ffmpeg.profile
# Firejail profile for ffmpeg
# Description: Tools for transcoding, streaming and playing of multimedia files
# This file is overwritten after every install/update
quiet
# Persistent local customizations
include ffmpeg.local
# Persistent global definitions
include globals.local

noblacklist ${MUSIC}
noblacklist ${VIDEOS}

include disable-common.inc
include disable-devel.inc
include disable-exec.inc
include disable-interpreters.inc
include disable-passwdmgr.inc
include disable-programs.inc
# include disable-shell.inc
include disable-xdg.inc

whitelist /usr/share/devedeng
whitelist /usr/share/ffmpeg
whitelist /usr/share/qtchooser
include whitelist-usr-share-common.inc
include whitelist-var-common.inc

apparmor
caps.drop all
ipc-namespace
machine-id
netfilter
nodvd
nogroups
nonewprivs
noroot
nosound
notv
nou2f
novideo
protocol inet,inet6
# allow set_mempolicy, which is required to encode using libx265
seccomp !set_mempolicy
shell none
tracelog

private-bin ffmpeg
private-cache
private-dev
# private-etc alternatives,ca-certificates,crypto-policies,hosts,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,nsswitch.conf,pkcs11,pki,resolv.conf,ssl
private-tmp

# dbus-user none
# dbus-system none

# memory-deny-write-execute - it breaks old versions of ffmpeg

to see if it works:
firejail --profile=~/.config/firejail/youtube-dl.profile /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'.
Tested, which failed with a new error.

firejail --profile=~/.config/firejail/youtube-dl.profile /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'
[youtube] 7RMz7tePA98: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno -3] Temporary failure in name resolution> (caused by URLError(gaierror(-3, 'Temporary failure in name resolution')))

You can remove the quiet from ffprobe,ffmpeg,ytdl profile to get a better overview of which profiles are being loaded by firejail...

Removed quiet.

hyiltiz@iPhone ~ $ firejail --profile=~/.config/firejail/youtube-dl.profile /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'
Reading profile /home/hyiltiz/.config/firejail/youtube-dl.profile
Reading profile /etc/firejail/allow-python2.inc
Reading profile /etc/firejail/allow-python3.inc
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-devel.inc
Reading profile /etc/firejail/disable-exec.inc
Reading profile /etc/firejail/disable-interpreters.inc
Reading profile /etc/firejail/disable-passwdmgr.inc
Reading profile /etc/firejail/disable-programs.inc
Reading profile /etc/firejail/disable-xdg.inc
Reading profile /etc/firejail/whitelist-usr-share-common.inc
Reading profile /etc/firejail/whitelist-var-common.inc
Parent pid 159161, child pid 159162
Warning: skipping httpd for private /etc
Warning: skipping apache for private /etc
Private /etc installed in 15.64 ms
41 programs installed in 72.33 ms
Blacklist violations are logged to syslog
Child process initialized in 186.68 ms
[youtube] 7RMz7tePA98: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno -3] Temporary failure in name resolution> (caused by URLError(gaierror(-3, 'Temporary failure in name resolution')))

Parent is shutting down, bye...
hyiltiz@iPhone ~ :( $ firejail --profile=NONE /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'Error: no profile with name "NONE" found.
hyiltiz@iPhone ~ :( $ firejail --profile=/dev/null /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'

If this still fails, try a combination of strace (strace -o file.txt /usr/bin/ytdl <link> and cat file.txt|egrep "/etc/|/usr/share/"

Failed, so trying strace.

hyiltiz@iPhone ~ $ strace -o dump.txt /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'
[youtube] 7RMz7tePA98: Downloading webpage
[youtube] 7RMz7tePA98: Downloading MPD manifest
[dashsegments] Total fragments: 15
[download] Destination: Firejail Introduction-7RMz7tePA98.f136.mp4
[download] 100% of 4.36MiB in 00:03
[dashsegments] Total fragments: 9
[download] Destination: Firejail Introduction-7RMz7tePA98.f140.m4a
[download] 100% of 1.14MiB in 00:01
[ffmpeg] Merging formats into "Firejail Introduction-7RMz7tePA98.mp4"
Deleting original file Firejail Introduction-7RMz7tePA98.f136.mp4 (pass -k to keep)
Deleting original file Firejail Introduction-7RMz7tePA98.f140.m4a (pass -k to keep)
hyiltiz@iPhone ~ $ rm Firejail\ Introduction-7RMz7tePA98.mp4 
hyiltiz@iPhone ~ $ cat dump.txt|egrep "/etc/|/usr/share/"
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
access("/etc/ld.so.preload", R_OK)      = 0
openat(AT_FDCWD, "/etc/ld.so.preload", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/localtime", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
stat("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en/LC_MESSAGES/messages.mo", 0x7fffa4dcb310) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
stat("/etc/mime.types", {st_mode=S_IFREG|0644, st_size=24546, ...}) = 0
openat(AT_FDCWD, "/etc/mime.types", O_RDONLY|O_CLOEXEC) = 3
stat("/etc/httpd/mime.types", 0x7fffa4dc88a0) = -1 ENOENT (No such file or directory)
stat("/etc/httpd/conf/mime.types", 0x7fffa4dc88a0) = -1 ENOENT (No such file or directory)
stat("/etc/apache/mime.types", 0x7fffa4dc88a0) = -1 ENOENT (No such file or directory)
stat("/etc/apache2/mime.types", 0x7fffa4dc88a0) = -1 ENOENT (No such file or directory)
stat("/usr/local/etc/httpd/conf/mime.types", 0x7fffa4dc88a0) = -1 ENOENT (No such file or directory)
stat("/usr/local/etc/httpd/conf/mime.types", 0x7fffa4dc88a0) = -1 ENOENT (No such file or directory)
stat("/usr/local/etc/mime.types", 0x7fffa4dc88a0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
stat("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd490) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd490) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcd490) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcd490) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd490) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd490) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en/LC_MESSAGES/messages.mo", 0x7fffa4dcd490) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en/LC_MESSAGES/messages.mo", 0x7fffa4dcd490) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcd1e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcd1e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en/LC_MESSAGES/messages.mo", 0x7fffa4dcd1e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en/LC_MESSAGES/messages.mo", 0x7fffa4dcd1e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en_US/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale/en/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/locale-langpack/en/LC_MESSAGES/messages.mo", 0x7fffa4dcd1a0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/youtube-dl.conf", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3
read(3, "# /etc/nsswitch.conf\n#\n# Example"..., 4096) = 494
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/host.conf", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/gai.conf", O_RDONLY|O_CLOEXEC) = 3
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3536, ...}) = 0
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 4

or under firejail: firejail --allow-debuggers --profile=profile strace -o file.txt /usr/bin/ytdl <link>)

This failed since strace wasn't found in the youtube-dl profile. Didn't want to further fuss with the profile we are trying to debug in the first place. But I'll do so if that is necessary.

hyiltiz@iPhone ~ $ firejail --allow-debuggers --profile=~/.config/firejail/youtube-dl.profile strace -o dump2.txt /usr/bin/ytdl 'https://www.youtube.com/watch?v=7RMz7tePA98'
Reading profile /home/hyiltiz/.config/firejail/youtube-dl.profile
Reading profile /etc/firejail/allow-python2.inc
Reading profile /etc/firejail/allow-python3.inc
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-exec.inc
Reading profile /etc/firejail/disable-interpreters.inc
Reading profile /etc/firejail/disable-passwdmgr.inc
Reading profile /etc/firejail/disable-programs.inc
Reading profile /etc/firejail/disable-xdg.inc
Reading profile /etc/firejail/whitelist-usr-share-common.inc
Reading profile /etc/firejail/whitelist-var-common.inc
Parent pid 160036, child pid 160037
Warning: skipping httpd for private /etc
Warning: skipping apache for private /etc
Private /etc installed in 20.98 ms
41 programs installed in 72.22 ms
Blacklist violations are logged to syslog
Child process initialized in 192.22 ms
Error: no suitable strace executable found

Parent is shutting down, bye...
hyiltiz@iPhone ~ :( $ which strace
/usr/bin/strace
hyiltiz@iPhone ~ $ firejail --allow-debuggers --profile=~/.config/firejail/youtube-dl.profile /usr/bin/strace -o dump2.txt /usr/bin/ytdl 'https://www.youtube.com/watch?v=7RMz7tePA98'
Reading profile /home/hyiltiz/.config/firejail/youtube-dl.profile
Reading profile /etc/firejail/allow-python2.inc
Reading profile /etc/firejail/allow-python3.inc
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-exec.inc
Reading profile /etc/firejail/disable-interpreters.inc
Reading profile /etc/firejail/disable-passwdmgr.inc
Reading profile /etc/firejail/disable-programs.inc
Reading profile /etc/firejail/disable-xdg.inc
Reading profile /etc/firejail/whitelist-usr-share-common.inc
Reading profile /etc/firejail/whitelist-var-common.inc
Parent pid 160094, child pid 160095
Warning: skipping httpd for private /etc
Warning: skipping apache for private /etc
Private /etc installed in 15.13 ms
41 programs installed in 63.22 ms
Blacklist violations are logged to syslog
Child process initialized in 179.90 ms
Error: no suitable /usr/bin/strace executable found

Parent is shutting down, bye...
hyiltiz@iPhone ~ :( $ firejail --allow-debuggers --profile=profile /usr/bin/strace -o dump2.txt /usr/bin/ytdl 'https://www.youtube.com/watch?v=7RMz7tePA98'
Error: no profile with name "profile" found.
hyiltiz@iPhone ~ :( $ firejail --allow-debuggers --profile=profile strace -o dump2.txt /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'
Error: no profile with name "profile" found.
hyiltiz@iPhone ~ :( $ firejail --allow-debuggers --profile=~/.config/firejail/youtube-dl.profile   strace -o dump2.txt /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'
Reading profile /home/hyiltiz/.config/firejail/youtube-dl.profile
Reading profile /etc/firejail/allow-python2.inc
Reading profile /etc/firejail/allow-python3.inc
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-exec.inc
Reading profile /etc/firejail/disable-interpreters.inc
Reading profile /etc/firejail/disable-passwdmgr.inc
Reading profile /etc/firejail/disable-programs.inc
Reading profile /etc/firejail/disable-xdg.inc
Reading profile /etc/firejail/whitelist-usr-share-common.inc
Reading profile /etc/firejail/whitelist-var-common.inc
Parent pid 160162, child pid 160163
Warning: skipping httpd for private /etc
Warning: skipping apache for private /etc
Private /etc installed in 16.70 ms
41 programs installed in 71.31 ms
Blacklist violations are logged to syslog
Child process initialized in 185.52 ms
Error: no suitable strace executable found

Parent is shutting down, bye...

and firejail --build /usr/bin/ytdl( should output info on terminal,if not look in /tmp for a file named firejail_straceXXX`) to see if something comes up specific to your system that isn't in any of the profiles.

This does seem to show interesting results but I am not sure what's specific to my system. It mentioned seccomp in the end, so provided a few filters for the mentioned logs. I'd also paste the strace dump but it is 20k LOC, and even pastebin is refusing to handle that.

$ firejail --build /usr/bin/youtube-dl  'https://www.youtube.com/watch?v=7RMz7tePA98'
[youtube] 7RMz7tePA98: Downloading webpage
[youtube] 7RMz7tePA98: Downloading MPD manifest
[dashsegments] Total fragments: 15
[download] Destination: Firejail Introduction-7RMz7tePA98.f136.mp4
[download] 100% of 4.36MiB in 00:02
[dashsegments] Total fragments: 9
[download] Destination: Firejail Introduction-7RMz7tePA98.f140.m4a
[download] 100% of 1.14MiB in 00:04
[ffmpeg] Merging formats into "Firejail Introduction-7RMz7tePA98.mp4"
Deleting original file Firejail Introduction-7RMz7tePA98.f136.mp4 (pass -k to keep)
Deleting original file Firejail Introduction-7RMz7tePA98.f140.m4a (pass -k to keep)
--- Built profile beings after this line ---
############################################
# /usr/bin/youtube-dl profile
############################################
# Persistent global definitions
# include /etc/firejail/globals.local

### basic blacklisting
include /etc/firejail/disable-common.inc
# include /etc/firejail/disable-devel.inc
include /etc/firejail/disable-passwdmgr.inc
# include /etc/firejail/disable-programs.inc

### home directory whitelisting
whitelist ~/youtube-dl.conf.txt
whitelist ~/youtube-dl.conf
include /etc/firejail/whitelist-common.inc

### filesystem

# private-tmp
# File accessed in /tmp directory:
# /tmp/firejail-strace.wHKbFJ,
private-dev
private-etc gcrypt,selinux,login.defs,passwd,hosts,youtube-dl.conf,mime.types,
blacklist /var
private-bin ffprobe,ffmpeg,bash,firejail,uname,python3.8,
# private-lib
blacklist /usr/share

### security filters
caps.drop all
nonewprivs
seccomp
# seccomp.keep wait4,futex,read,mmap,write,openat,stat,close,poll,fstat,mprotect,ioctl,getdents64,execve,lseek,unlink,connect,clone,socket,getsockopt,munmap,sendmmsg,rename,recvmsg,recvfrom,setsockopt,getsockname,getpid,sendto,brk,rt_sigaction,access,lstat,utimensat,setresuid,setresgid,getuid,readlink,bind,getpeername,rt_sigprocmask,geteuid,getgid,pipe2,umask,prlimit64,arch_prctl,set_robust_list,getcwd,select,sysinfo,uname,set_tid_address,statfs,getegid,getppid,getpgrp,mlock,sched_getaffinity,fcntl,getrandom,prctl,getrusage,gettid,madvise,dup,dup2,sigaltstack
# 68 syscalls total
# Probably you will need to add more syscalls to seccomp.keep. Look for
# seccomp errors in /var/log/syslog or /var/log/audit/audit.log while
# running your sandbox.

### network
protocol inet,inet6,
# net eth0
netfilter

### environment
shell none
hyiltiz@iPhone ~ $ ^C
hyiltiz@iPhone ~ :( $ rg seccomp /var/log/syslog 
414:Sep 21 14:28:51 iPhone kernel: [    0.107616] Spectre V2 : User space: Mitigation: STIBP via seccomp and prctl
416:Sep 21 14:28:51 iPhone kernel: [    0.107617] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
1506:Sep 21 14:28:53 iPhone chronyd[1703]: Loaded seccomp filter
hyiltiz@iPhone ~ $ rg seccomp /var/log/audit/audit.log
hyiltiz@iPhone ~ :( $ exit

@rusty-snake
Copy link
Collaborator

What the state here?

@hyiltiz
Copy link
Author

hyiltiz commented Nov 9, 2020

I've been getting help trying to pinpoint a profile that works and why it wasn't working. Still awaiting help.

@bbhtt
Copy link
Contributor

bbhtt commented Nov 11, 2020

Try adding this to the private-etc line of youtube-dl.profile separated by commas, remove if one is duplicate, "ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,httpd,apache,apache2,nsswitch.conf,host.conf,gai.conf"

Did that.

Sorry if there was a confusion but you should've added it to the original private-etc line not make a new line like this:

# private-etc alternatives,ca-certificates,crypto-policies,hostname,hosts,ld.so.cache,mime.types,pki,resolv.conf,ssl,youtube-dl.conf
private-etc ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,httpd,apache,apache2,nsswitch.conf,host.conf,gai.conf private-tmp

So only one private-etc like this:

private-etc alternatives,apache,apache2,ca-certificates,crypto-policies,gai.conf,hostname,hosts,host.conf,httpd,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,mime.types,nsswitch.conf,pki,resolv.conf,ssl,youtube-dl.conf

Anyways looking at the strace and build I don't think even above will work and in that case I'm out of ideas; sorry.

@hyiltiz
Copy link
Author

hyiltiz commented Nov 11, 2020

Huh? There was no extra newlines in my tests in private-etc line; the first line is commented out with #, and the second line, IIRC, should be one I copied out from this thread.

Shall we keep this open to see if someone else may have some ideas? @kortewegdevries couldn't replicate it, so maybe waiting until someone else could replicate would be meaningful too.

@bbhtt
Copy link
Contributor

bbhtt commented Nov 11, 2020

Huh? There was no extra newlines in my tests in private-etc line; the first line is commented out with #, and the second line, IIRC, should be one I copied out from this thread.

You commented the original private-etc line from the profile and added only this private-etc ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,httpd,apache,apache2,nsswitch.conf,host.conf,gai.conf.

Hence the error in your post: (because with only that you are missing hosts,resolv.conf, etc.)

ERROR: Unable to download webpage: <urlopen error [Errno -3] Temporary failure in name resolution> (caused by URLError(gaierror(-3, 'Temporary failure in name resolution')))

I said to combine both like this: private-etc alternatives,apache,apache2,ca-certificates,crypto-policies,gai.conf,hostname,hosts,host.conf,httpd,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,mime.types,nsswitch.conf,pki,resolv.conf,ssl,youtube-dl.conf not comment the original one and add another line.

@hyiltiz
Copy link
Author

hyiltiz commented Nov 11, 2020

Aha, gotcha! Thanks for patiently explaining. Did with only changing youtube-dl profile and the ffmpeg profile, and then also (the system-wide):

yiltiz@iPhone ~/Downloads [1]> grep private-etc ~/.config/firejail/ffmpeg.profile 
# private-etc alternatives,ca-certificates,crypto-policies,hosts,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,nsswitch.conf,pkcs11,pki,resolv.conf,ssl
private-etc alternatives,apache,apache2,ca-certificates,crypto-policies,gai.conf,hostname,hosts,host.conf,httpd,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,mime.types,nsswitch.conf,pki,resolv.conf,ssl,youtube-dl.conf
hyiltiz@iPhone ~/Downloads> grep private-etc ~/.config/firejail/youtube-dl.profile 
# private-etc alternatives,ca-certificates,crypto-policies,hostname,hosts,ld.so.cache,mime.types,pki,resolv.conf,ssl,youtube-dl.conf
# private-etc ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,httpd,apache,apache2,nsswitch.conf,host.conf,gai.conf
private-etc alternatives,apache,apache2,ca-certificates,crypto-policies,gai.conf,hostname,hosts,host.conf,httpd,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,mime.types,nsswitch.conf,pki,resolv.conf,ssl,youtube-dl.conf
hyiltiz@iPhone ~/Downloads> rm Firejail\ Introduction-7RMz7tePA98.f1*
hyiltiz@iPhone ~/Downloads> grep private-etc /etc/firejail/ffmpeg.profile
# private-etc alternatives,ca-certificates,crypto-policies,hosts,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,nsswitch.conf,pkcs11,pki,resolv.conf,ssl
private-etc alternatives,apache,apache2,ca-certificates,crypto-policies,gai.conf,hostname,hosts,host.conf,httpd,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,localtime,mime.types,nsswitch.conf,pki,resolv.conf,ssl,youtube-dl.conf
hyiltiz@iPhone ~/Downloads> firejail --profile=~/.config/firejail/youtube-dl.profile /usr/bin/youtube-dl 'https://www.youtube.com/watch?v=7RMz7tePA98'
Reading profile /home/hyiltiz/.config/firejail/youtube-dl.profile
Reading profile /etc/firejail/allow-python2.inc
Reading profile /etc/firejail/allow-python3.inc
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-devel.inc
Reading profile /etc/firejail/disable-exec.inc
Reading profile /etc/firejail/disable-interpreters.inc
Reading profile /etc/firejail/disable-passwdmgr.inc
Reading profile /etc/firejail/disable-programs.inc
Reading profile /etc/firejail/disable-xdg.inc
Reading profile /etc/firejail/whitelist-usr-share-common.inc
Reading profile /etc/firejail/whitelist-var-common.inc
Parent pid 1140388, child pid 1140389
47 programs installed in 100.35 ms
Warning fcopy: skipping /etc/alternatives/fakeroot, cannot find inode
Warning fcopy: skipping /etc/alternatives/cssparse, cannot find inode
Warning fcopy: skipping /etc/alternatives/vim, cannot find inode
Warning fcopy: skipping /etc/alternatives/display-im6, cannot find inode
Warning fcopy: skipping /etc/alternatives/wine.fr.1.gz, cannot find inode
Warning fcopy: skipping /etc/alternatives/wineconsole, cannot find inode
Warning fcopy: skipping /etc/alternatives/pinentry-x11, cannot find inode
Warning fcopy: skipping /etc/alternatives/proxychains, cannot find inode
Warning fcopy: skipping /etc/alternatives/winepath, cannot find inode
Warning fcopy: skipping /etc/alternatives/sar, cannot find inode
Error fcopy: size limit of 500 MB reached
Warning: skipping apache for private /etc
Warning: skipping crypto-policies for private /etc
Warning: skipping youtube-dl.conf for private /etc
Private /etc installed in 43.88 ms
Blacklist violations are logged to syslog
Child process initialized in 270.84 ms
[youtube] 7RMz7tePA98: Downloading webpage
[youtube] 7RMz7tePA98: Downloading MPD manifest
[dashsegments] Total fragments: 15
[download] Destination: Firejail Introduction-7RMz7tePA98.f136.mp4
[download] 100% of 4.36MiB in 00:03
[dashsegments] Total fragments: 9
[download] Destination: Firejail Introduction-7RMz7tePA98.f140.m4a
[download] 100% of 1.14MiB in 00:01
[ffmpeg] Merging formats into "Firejail Introduction-7RMz7tePA98.mp4"
ERROR: ffmpeg: error while loading shared libraries: libGL.so.1: cannot open shared object file: No such file or directory

Parent is shutting down, bye...

@rusty-snake
Copy link
Collaborator

I'm closing here due to inactivity, please fell free to request to reopen if you still have this issue.

@hyiltiz
Copy link
Author

hyiltiz commented Jan 4, 2021

I am willing to offer any help in debugging as I've been doing, but cannot resolve the issue alone by myself and the issue still persists.

@rusty-snake rusty-snake reopened this Jan 5, 2021
@rusty-snake
Copy link
Collaborator

I can't follow this thread. Can you give a short summary.

@hyiltiz
Copy link
Author

hyiltiz commented Feb 5, 2021

Ok, I'll try to summarize.

  • youtube-dl and ffmpeg works fine without firejail
  • it seems some specific list in private-etc and/or private-bin for them both may resolve this issue, but unsure what (this was ongoing investigation and I am still awaiting response to my test above)
  • libGL.so.1 seems to be the culprit in most of these cases, and it seems not only specific to ffmpeg or youtube-dl, as I can reproduce the same issue with okular, despite having installing pakcages that provide libGL.so.1
  • I am running a x86-64 Debian but also installed i386 packages (Steam and some packages only provide a 32bit build...), and am wondering if that is the root cause. Is it possible that firejail, detecting my system as x86-64, didn't bothered to properly mask the i386 libraries, and the x86-64 binaries accidentally found the i386 libraries when invoked with firejail which masked x86-64? (showing okular below as youtube-dl calls other programs and makes debugging too complicated)
# ll /usr/lib/x86_64-linux-gnu/libGL.so.1
Permissions Size User Date Modified Name
lrwxrwxrwx    50 root  3 Nov  2020  /usr/lib/x86_64-linux-gnu/libGL.so.1 -> /etc/alternatives/glx--libGL.so.1-x86_64-linux-gnu
~# ll /usr/lib/i386-linux-gnu/libGL.so.1
Permissions Size User Date Modified Name
lrwxrwxrwx    48 root  3 Nov  2020  /usr/lib/i386-linux-gnu/libGL.so.1 -> /etc/alternatives/glx--libGL.so.1-i386-linux-gnu
# ldd /usr/bin/okular|rg libGL
        libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007f83aecb9000)
        libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007f83ade49000)
        libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007f83ade15000)

In all cases, directly calling the program with something like /usr/bin/okular or /usr/bin/ffmpeg will resolve the issue, unless that command then calls another program that has a firejail profile (like youtube-dl calls ffmpeg which has a profile). I'd rather not throw away the firejail profiles all the time, but not sure how to work around it.

I have apparmor running, although none of the mentioned programs above are listed in aa-status, so not sure if apparmor is relevant.

@rusty-snake
Copy link
Collaborator

didn't bothered to properly mask the i386 libraries, and the x86-64 binaries accidentally found the i386 libraries when invoked with firejail which masked x86-64?

blacklist /usr/lib/i386-linux-gnu (or where else debian has i386 libs) can be used to test this.

it seems some specific list in private-etc and/or private-bin for them both may resolve this issue, but unsure what (this was ongoing investigation and I am still awaiting response to my test above)

So if you add ignore private-bin and ignore private-etc to ffmpeg.local and youtube-dl.local it works?

@hyiltiz
Copy link
Author

hyiltiz commented Feb 7, 2021

So if you add ignore private-bin and ignore private-etc to ffmpeg.local and youtube-dl.local it works?

Yes. I'd rather not ignore them, so I think we've been bisecting a list of things to ignore above.

blacklist /usr/lib/i386-linux-gnu (or where else debian has i386 libs) can be used to test this.

Added blacklist /usr/lib/i386-linux-gnu to okular.local but still gives the same error message. Guess the assumption wasn't correct.

@rusty-snake
Copy link
Collaborator

So if you add ignore private-bin and ignore private-etc to ffmpeg.local and youtube-dl.local it works?

Yes. I'd rather not ignore them, so I think we've been bisecting a list of things to ignore above.

That right, but no narrow it down I asked if it works if they are ignored.

Assuming that adding only one of the ignores still breaks (i.e. both are necessary):

  • Does ignore private-etc and private-bin ldconfig work?
  • Does it work with ignore private-bin and private-etc <ALL FILES>?
    Use unalias ls && ls /etc | tr '\n' ',' to get a list of all files/dirs. You will first get some really nothing saying error messages from firejail on files like sudoers, remove all such files until ytdl starts. If it works now, remove files from the list until you find the most minimal list which works.

@1ras
Copy link

1ras commented Aug 25, 2023

Regarding libblas.so, this is because "private-etc alternatives" provides now an incomplete alternatives directory (this was not the case in the past):

Native system:

$ ls -l /usr/lib/x86_64-linux-gnu/libblas.so.3
lrwxrwxrwx 1 root root 47  2. Sep 2019  /usr/lib/x86_64-linux-gnu/libblas.so.3 -> /etc/alternatives/libblas.so.3-x86_64-linux-gnu
$ ls -l /etc/alternatives/libblas.so.3-x86_64-linux-gnu
lrwxrwxrwx 1 root root 55 22. Aug 22:48 /etc/alternatives/libblas.so.3-x86_64-linux-gnu -> /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
$ ls -l /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
-rw-r--r-- 1 root root 399704 19. Dez 2022  /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3

Inside Firejail:

$ ls -l /usr/lib/x86_64-linux-gnu/libblas.so.3
lrwxrwxrwx 1 nobody 65534 47  2. Sep 2019  /usr/lib/x86_64-linux-gnu/libblas.so.3 -> /etc/alternatives/libblas.so.3-x86_64-linux-gnu
$ LANG=C  ls -l /etc/alternatives/libblas.so.3-x86_64-linux-gnu
ls: cannot access '/etc/alternatives/libblas.so.3-x86_64-linux-gnu': No such file or directory
$ ls -l /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
-rw-r--r-- 1 nobody 65534 399704 19. Dez 2022  /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3

Same issue with wine executables:

Native system:

$ ls -l /usr/bin/wine
lrwxrwxrwx 1 root root 22 18. Feb 2023  /usr/bin/wine -> /etc/alternatives/wine
$ ls -l /etc/alternatives/wine
lrwxrwxrwx 1 root root 20 18. Feb 2023  /etc/alternatives/wine -> /usr/bin/wine-stable
$ ls -l /usr/bin/wine-stable
-rwxr-xr-x 1 root root 1029 18. Feb 2023  /usr/bin/wine-stable

Inside Firejail:

$ ls -l /usr/bin/wine
lrwxrwxrwx 1 nobody 65534 22 18. Feb 2023  /usr/bin/wine -> /etc/alternatives/wine
$ LANG=C ls -l /etc/alternatives/wine
ls: cannot access '/etc/alternatives/wine': No such file or directory
$ ls -l /usr/bin/wine-stable
-rwxr-xr-x 1 nobody 65534 1029 18. Feb 2023  /usr/bin/wine-stable

For some reason /etc/alternatives is incomplete with "private-etc alternatives". Some links are still available, others are missing. I can see no good reason why symlinks to installed software are "randomly" removed.

This affects Debian Bookworm, firejail 0.9.72.

@kmk3
Copy link
Collaborator

kmk3 commented Aug 26, 2023

Regarding libblas.so, this is because "private-etc alternatives" provides now
an incomplete alternatives directory (this was not the case in the past):

This may potentially be fixed by:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants