[🐧] GNU+Linux compatibility progress

[YoshiRulz]

jump to known bugs

Overview:

• Nothing on this page applies to the (discontinued) macOS port.
  • Also, EmuHawk is only for x86-64 machines for the time being. If you have an x86 or AArch64 machine, you should have no trouble installing the rundeps, but EmuHawk will crash on start; see DllNotFoundException: libbizhash when opening any rom on Linux AArch64 #3687. Contributions welcome.
• Build instructions are in the readme. tl;dr: run Dist/BuildRelease.sh.
• "Installation" instructions are also in the readme. tl;dr: run output/EmuHawkMono.sh.
  • No Wine required! If you get it working in Wine, good for you, but I'm going to ignore you and hide any counter-productive comments.
• 40-ish of 45-ish systems can be emulated, with the notable exception of N64 (Mupen64Plus isn't available, and the experimental Ares64 core is too slow).
  • But this may soon be resolved with [WIP] Update mupen64plus to latest upstream version #4117!
• For casual players, most things work as they would on Windows.
• For TASers, most things work as they would on Windows.
• For glitch hunters and romhackers, everything sucks and I'm sorry. But if a core is available on Linux, and the core implements the necessary services for a tool, then it should be at least functional.
• Before 2.10:
  • Command-line flags to EmuHawkMono.sh are passed through as expected. The single exception is if you pass --mono-no-redirect as the first flag, which disables redirecting stdout/stderr to EmuHawkMono_last*.txt (the default behaviour is to write to disk instead of the terminal).
  • When it inevitably crashes, some errors go to the terminal/EmuHawkMono_last*.txt, some appear in dialog boxes, some both. Most of these error dialogs will either have a "Copy" button, or will allow Ctrl+C.

---

As of 2.10.1, the following systems can be emulated:

• 3DO with Opera (I don't have roms --yoshi)
• 3DS with Encore
• Amiga with UAE
• Apple II with Virtu
• Arcades with MAME
• Atari 2600 with Atari2600Hawk or Stella
• Atari 7800 with A7800Hawk
• Atari Jaguar with VirtualJaguar
• Atari Lynx with Handy
• Channel F with ChannelFHawk (I don't have roms --yoshi)
• ColecoVision with ColecoHawk
• Commodore 64 with C64Hawk (loads to BASIC prompt, at least --yoshi)
• DOOM with DSDA-Doom
• Game Boy and GBC with Gambatte, GBHawk, or SameBoy
• GBA with mGBA
• Intellivision with IntelliHawk
• Magnavox Odyssey² with O2Hawk
• MS-DOS/Win9x with DOSBox-X (I don't have roms --yoshi)
• MSX with MSXHawk
• N64 with Ares64
• N64DD with Ares64 (I don't have roms --yoshi)
• Neo Geo Pocket / Color with NeoPop
• NES/Famicom plus FDS with NesHawk or QuickNes
• Nintendo DS with melonDS
• Nintendo Virtual Boy with Virtual Boyee
• PC Engine / TG-16 and SuperGrafx plus CD with HyperNyma, PCEHawk, or TurboNyma
• PC-FX with T.S.T.
• PSX with Nymashock
• Sega 32X with PicoDrive
• Sega CD with Genplus-gx
• Sega Game Gear with SMSHawk
• Sega Genesis / Mega Drive with Genplus-gx
• Sega Master System with SMSHawk
• Sega Pico with Genplus-gx (I don't have roms --yoshi)
• Sega Saturn with Saturnus
• Sega SG-1000 SMSHawk
• SNES/SFC with BSNES, Faust, or Snes9x
• Super Game Boy with Gambatte or BSNES
• TI-83 with Emu83 or TI83Hawk (see "Issues with TI83Hawk" below)
• TIC-80 fantasy console with TIC-80 reference implementation
• Uzebox fantasy console with Uzem
• Vectrex with VectrexHawk
• WonderSwan / Color with Cygne
• ZX Spectrum with ZXHawk
• Anything else via Libretro cores (provided they don't crash)

[YoshiRulz]

Issues with EmuHawk

Startup issues:

• Gtk not found (missing LD_LIBRARY_PATH to libgtk-x11-2.0.so.0?), using built-in colorscheme—see ignored GTK theme below.
• Unable to locate theme engine in module_path: "adwaita" seems to be a harmless error, but can be resolved by installing gnome-themes-extra.

Misc. issues:

• Mono seemingly ignores the GTK theme... on my machine >:( (Others have reported mono/mono#20078 dark theme being inconsistently applied). As a result, everything is an ugly beige, and some icons in e.g. open file dialogs are missing. Hope you remember which is the parent directory button!
  • THEMING UPDATE 2024-09: Some work has been put into making the default not so ugly, but hopefully the "correct" light theme can be installed and coerced to load as was done for NixOS in 9a2c163. Users who were affected by mixed dark/light theme have had success with the GTK_DATA_PREFIX='' ./EmuHawkMono.sh workaround, so perhaps that could be applied by default.
    • GTK_DATA_PREFIX seems to have no effect on Ubuntu 24.04 (GNOME).
  • There was a tiny amount of work put into making the mixed dark/light theme usable (b967e60), but EmuHawk was designed for a light theme. Please report if you're affected by the mixed dark/light theme, but don't report individual instances of illegible text for example—those are wontfix.
• I think file extensions are case-sensitive on Mono. The rom loader dialog doesn't recognise my .32X file as a rom. fixed in f8a688d
• (unthrottled) FPS triples when window unfocused. Is input slowing it down? who knows --yoshi
  • actually I can't repro this --yoshi at 25a5449
  • dwango repro'd this on stream. still no idea why it happens. --yoshi
  • Still can't repro, no change when swapping out OpenTKInputAdapter for a DummyInputAdapter --yoshi at 24f69eb
  • Repro'd on NixOS, both on 2.9.1 and at ebe3cad, though it's a very small difference in Release config. --yoshi
  • It's just defocusing MainForm, by the way, you can have the Lua Console or Virtual Pad focused to get the speedup
• Keyboard shortcuts in RAM Watch don't work. ToolStripMenuItem.ShortcutKeys isn't working at all.
  • Just looked into this again with TAStudo and ProcessCmdKey isn't even being called. --yoshi at 681ef72
• Menubar mnemonics are disabled. Past me did it to prevent a crash, needs retesting. --yoshi at 03d10ce
  • enabled on MainForm as of 28d62e6
  • not working on other Forms --yoshi at 28d62e6
  • probably same cause as ShortcutKeys --yoshi at 681ef72
• Some file open dialogs cause a ArgumentOutOfRangeException in System.DateTimeOffset.FromUnixTimeSeconds(long). --yoshi at 03d10ce
  • tracked as [WinForms] Unable to browse using OpenFileDialog without crashing where subdir has "out-of-range" timestamp mono/mono#21213
  • note Paths config uses FolderBrowserDialog which seems to crash regardless of the timestamps' validity
• In IToolFormAutoConfigs, Settings > Stay on Top correctly writes Window.KeepAbove = !item.Checked, but does not read the actual value i.e. toggling keep-above via the WM does not change the checkbox's state. I suggest this menu item be disabled on Linux (and macOS?). --yoshi at 03d10ce
  • Fixed in 329a7de (by hiding the menu item).
• Defaults for Config > Paths... could be changed to standards like $XDG_CONFIG_HOME. Making this change to system-global dirs would break from the "portable app" precedent. --yoshi
  • Sorta done in NixHawk
• Users with only an Intel iGPU (i.e. no discrete GPU) may get i965: Failed to submit batchbuffer: Invalid argument. Try LIBGL_ALWAYS_SOFTWARE=1 ./EmuHawkMono.sh or otherwise setting that environment variable. (workaround found by @InfamousKnight)
• Dialogs which pause emulation should cause the current frame to be redrawn to MainForm, but instead drawing stops and you get the solitaire effect. --yoshi at 268ab96
• Possibly related, under Plasma Wayland on NixOS: "open file" dialogs sometimes result in empty black windows, which then become "infectious" and result in even the main window only showing black, which persists until user logout / compositor shutdown. --bugQ
• Dropdowns not closing: WinForms - ToolStrip dropdowns do not close properly mono/mono#12644 --yoshi at 42fc895
  • workaround added in 653844e (sorry I didn't do that sooner --yoshi)
• Controls in RecordMovie (dialog asking for filename and author for new movie) are misplaced, making keyboard navigation a requirement for recording movies (without TAStudio) --yoshi at 9e60927
  • Fixed in d32f734.
• If you get an error like "GDI+ status: InvalidParameter" at System.Drawing.Image.get_FrameDimensionsList, try updating libgdiplus.
• Hard crash when clicking+dragging scrollbar in OpenFileDialog. Nothing in stdout or stderr. ¯\_(ツ)_/¯
  • Similarly, random hard crashes when clicking+dragging scrollbar in core settings dialogs, then moving cursor over the button of a dropdown menu without releasing the mouse button.
• In Hex Editor, opening context menu causes scroll wheel and arrow keys to have no effect until the window is unfocused and refocused.
• Log Window threading is fragile, see LogWindow something causes crash on Mono #2741.
• Starting Mono while the solution is being rebuild will, rather than hitting the version check in EmuHawk.Program, cause Mono to segfault. (Not before init'ing GTK, though. I swear it's taunting me.)
• May be NixOS-specific, but it seems idling with Lua scripts running+drawing but emulation paused causes the Lua Console (possible all WinForms UI?) to freeze, and unpausing emulation fixes it immediately...
  • It also seems to unpause for no reason? Does it think I closed a menu and it was only temp. paused?
• RetroAchievements dialogs aren't great, but they are technically usable
• From 2.10, SDL2 handles sleep prevention, but while it's implemented for Linux (X11), it's not working in practice. See BizHawk does not inhibit display or system sleep while running #3644 (comment).
  • possibly related: melonDS appears as "My SDL application" to KDE Plasma power management melonDS-emu/melonDS#2300
• PictureBox applies anti-aliasing/interpolation to pixel graphics: [🐧] NES PPU Viewer's "zoom box" shouldn't be anti-aliased #3764
• Buttons in some windows and dialogs are missing their labels. Possibly related to Japanese locale. Initially reported by @miyakut on Discord, running Mint. 404f28f didn't work and was reverted.
• Lua Console REPL, and possibly elsewhere, doesn't accept numpad input. Repro'd on Ubuntu 22.04 but not NixOS.

Cores

Issues with TI83Hawk

• IndexOutOfRangeException at BizHawk.Emulation.Cores.Calculators.TI83.ReadMemory(ushort) --yoshi at 25a5449

Building and packaging

Dependencies are being recorded (so build scripts can be made) in the GitLab mirror's snippets. On multi-arch distros, target x86_64/amd64.

For the record I've mostly focused on Manjaro, Nix/NixOS (which I use), Linux Mint, and Ubuntu. --yoshi

Bus error (core dumped) when building may be because you're out of disk space.

libblip_buf works as intended on both OSes, but the Makefile for Linux builds was hacked together (see libblip_buf issue tracker) and using the library in BizHawk was intended to be a temporary hack. The library has been rewritten as noted in #1312. Sappharad has previously attempted moving it to managed C#. It may be possible to use libspeex instead. they serve different purposes according to #3207 (comment) --yoshi

Don't know where else to leave these:

• Debug P/Invokes (.dll/.so loading) with MONO_LOG_LEVEL=debug MONO_LOG_MASK=dll ./EmuHawkMono.sh --mono-no-redirect.
• It seems libgdiplus.so doesn't support whatever the "PNG 256x256" mode of .icos is.

[nattthebear]

For Gambatte et al, I do not think it will be that difficult to compile these cores for Bizhawk. The source code for the version that is used in Bizhawk is in the Bizhawk repo, and most/all of them are compiled with mingw right now, so don't have too many MSVCisms in them. You should give one a try.

For the waterbox cores, I think it shouldn't be too hard to write a version of MemoryBlock that works with linux APIs. I'm willing to give it a try if no one else is.

[Sappharad]

@Sappharad Could you please verify that A7800Hawk, C64Hawk, ColecoHawk, GBHawk, IntelliHawk, NesHawk, PCEHawk, SMSHawk, and ZXHawk work on macOS?

As of the last time I pulled changes from master, (January 1st) A7800, ColecoHawk, GBHawk, Gambatte, NESHawk, QuickNES, PCEHawk & SMS Hawk are currently working on macOS. I have not tested any other cores at this time, some will require preparations for me to use them as I have never used them. If you're asking about the latest code, I'll have to rebase tomorrow. Note that I also have a bunch of changes that differ from yours, but nothing on in the backend should be different so I'm not worried about C# cores not working.

[alyosha-tas]

Honestly, I can probably just remove the BG color setting from Atari2600. That BG color is outside of the frame, if anything it should be a global setting. Most other cores just set it to black with no option to change it.

[YoshiRulz]

@Sappharad So you had no issues building libblip_buf? And how did you get Gambatte and QuickNES working? Re: ZX Spectrum Asnivor told me how to use it

[Asnivor]

@alyosha-tas Changing background colour to a global setting may be problematic for the cores that do enable the user to set the colour. For example, ZXHawk has the ability to set this colour through it's options (as this was a feature request by someone in the past). However, the reflection related problems are not present as the choice of colours is limited to only the ones that the emulated machine can display (and it makes its journey through Settings as a simple integer lookup into the core's palette array)

Obviously cores that specifically manage this setting will automatically (and invisibly) override any new global background colour implementation. As long everyone is happy with this (somewhat counter-intuitive from a user-perspective) behavior, global should work.

[YoshiRulz]

@Asnivor to add to that, I suggested in #1380 to use an integer everywhere except actually drawing the background (which ZXHawk already does?), using Color.FromArgb(int) (MSDN).

[Asnivor]

@YoshiRulz yup, that will solve the issues. Unfortunately a lot of the cores use the auto-generated settings menus, so passing a Color reference gives you an easy color picker

I guess the auto generated settings dialog can be modified to accept int values and then use Color.FromArgb. Thats probably the easiest solution (after removing the setting from problem cores entirely). ZXHawk is slightly different in that the integer it passes is never more than 15 (as this is the max number of colours that computer had and this is used to lookup a colour palette array in-core). But integer representations argb should work if someone wants to make that change.

[nattthebear]

Has anyone looked into the DefaultValue Color crash in detail? It doesn't seem like something that should be broken, and I've confirmed that [DefaultValue(typeof(Color), "Black")] does work in a basic mono repro.

[Sappharad]

@Sappharad So you had no issues building libblip_buf? And how did you get Gambatte and QuickNES working? Re: ZX Spectrum Asnivor told me how to use it

I had already ported them a couple years ago in the mono-portable branch. It was just a matter of copying the old Xcode projects from the mono-portable branch and switching them to x64. Anything I've done before I've been slowly grabbing pieces from the branch and re-implementing in my fork. In some cases it differs from your approach and I've just been keeping mine. Keyboard for example, you built a massive mapping between SlimDX and OpenTK, but I don't even reference SlimDX in mine because that reference is conditional to Windows so the mono build acts like it's not even there. Then I just use the OpenTK Keyboard enum directly. (And there's a separate default mappings file for OpenTK where I just did a find/replace on the key names that differ like Keys.Return becomes Keys.Enter instead)

I'll try to rebase later today and test again, but I'd say don't worry about it too much because if it works for you I'm not worried about it working for me.

Oh, actually, there's one thing you need to change once you have a QuickNES build for Linux if you want it to work. You need to conditionally take out the calls for the Windows floating point mode hack. I tested it several years ago, natt or someone explained what bug that was there to fix and I found the bug didn't occur at all macOS. So there wasn't a need to replicate whatever that does. Example is here:
Sappharad@2bcc546#diff-a9975d57b1522ccb872d9e6dc6f1f171
Note that this won't work for you as-is, because you've been using runtime checks everywhere for platform while I do it as preprocessor directives, but you can do something similar your way. Note that if you'd rather use the preprocessor directives for this case that some of the projects like BizHawk.Client.Common have a WINDOWS define hard-coded in the project settings instead of being conditionally defined for Windows only. Those are also fixed on my end and I can probably do a diff and tell you which ones are broken.

[Asnivor]

@nattthebear

Has anyone looked into the DefaultValue Color crash in detail? It doesn't seem like something that should be broken, and I've confirmed that [DefaultValue(typeof(Color), "Black")] does work in a basic mono repro.

So i've had a rum ol' time time of spinning up a linux vm, installing mono, building bizhawk and attempting to debug in the buggy shitshow that IS MonoDevelop.

The line where it all goes wrong is here: https://github.com/TASVideos/BizHawk/blob/7ff08fb63e4f96cb55ba4c481d59c2dc0a6fbde9/BizHawk.Common/SettingsUtil.cs#L65

In Windows, PropertyInfo.GetCustomAttributes(true) successfully returns the System.Drawing.Color value:

In Mono however, the same PropertyInfo method fails to return the System.Drawing.Color value:

To my untrained eye, both PropertyInfo objects appears to have much of the correct info:

However the locals window in monodevelop doesn't provide a lookup into the CustomAttributes IEnumerable (for whatever IDE-centric reason).

I suspect this is all some issue/edge case with the way Mono handles reflection, but I really don't know enough to go much deeper. .GetCustomAttributes() is working fine in Mono for attribute values that do not involve the possible 'double reflection' that this BackgroundColor property does.

So yeah, not sure what else I can try...

[nattthebear]

I tried the following small example against mono and .net and it worked against both of them. I wonder what's different in the full case? Not sure what you mean about "double reflection".

class Foo
{
    [System.ComponentModel.DefaultValue(typeof(System.Drawing.Color), "Black")]
    public System.Drawing.Color Bar { get; set; }
    static void Main()
    {
        var prop = typeof(Foo).GetProperty("Bar");
        var attr = (System.ComponentModel.DefaultValueAttribute)prop.GetCustomAttributes(true)[0];
        var value = attr.Value;
        if (value == null)
        {
            System.Console.WriteLine("value is null");
        }
        else if (value.GetType() != typeof(System.Drawing.Color))
        {
            System.Console.WriteLine("type is " + value.GetType());    
        }
        else
        {
            System.Console.WriteLine("color is `" + value + "`");
        }
    }
}

[Asnivor]

Hmmm.

Issue is there for me with your code. Are you running Mono on windows or linux?

If it matters, I am running:

• Debian Stretch
• latest stable mono-devel & mono-dbg
• latest stable monodevelop

IIRC, @YoshiRulz was seeing the same issue on Arch linux

[YoshiRulz]

@nattthebear I'm also getting "value is null" on Manjaro w/ Mono, Mono VB.NET:

~$> mono --version
Mono JIT compiler version 5.16.0 (makepkg/bb3ae37d71a Sat Dec  8 13:21:35 CET 2018)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug 
        Interpreter:   yes
        LLVM:          supported, not enabled.
        GC:            sgen (concurrent by default)
~$> yay -Qi mono-basic | head -n2
Name            : mono-basic
Version         : 4.7-1
~$> lsb_release -a
LSB Version:    n/a
Distributor ID: ManjaroLinux
Description:    Manjaro Linux
Release:        18.0.2
Codename:       Illyria

[Asnivor]

Right @nattthebear is going to check his end again today, but if there is a discrepancy lets maybe lets get some version details in here just in case this is something we think about raising with Mono.

For me:

~$ mono --version
Mono JIT compiler version 5.18.0.255 (tarball Wed Jan  2 21:21:16 UTC 2019)
	TLS:		__thread
	SIGSEGV:	altstack
	Notifications:	epoll
	Architecture:	amd64
	Disabled:	none
	Misc:		softdebug
	Interpreter:	yes
	LLVM:		yes(600)
	Suspend:	preemptive
	GC:		sgen (concurrent by default)

~$ lsb_release -a
No LSB modules are available
Distributor ID:	Debian
Description:	Debian GNU/Linux 9.6 (stretch)
Release:	9.6
Codename:	stretch