Is your feature request related to a problem?

Streaming trajectories directly into MDAnalysis is one of the MDA 3.0 milestones. Specifically, streaming a trajectory directly from a simulation engine into MDA enables a few keys things that aren't otherwise possible:

• Performing high time resolution analyses without generating massive trajectory files (for example, measuring ions crossing a membrane)
• Writing only a sub-selection of the system into a trajectory file with an MDA writer (not a built-in feature in all simulation engines), removing an intermediate step
• (In the future) Inputting forces calculated in Python with MDAnalysis back into the simulation engine to be applied to atoms in the simulation

The Interactive MD (IMD) version 3 protocol, which builds on the IMDv2 protocol already implemented in GROMACS, NAMD, and LAMMPS, is the most effective way to achieve this "in-situ" analysis since it requires only minimal changes from the working IMDv2 implementations made to work with VMD.

MDAnalysis can quickly gain the ability to read a trajectory via this protocol using the reader API by adopting the code in imdclient, an MDAKit built in a collaboration between @orbeckst and @HeydenLabASU.

Already, we have a working implementation in GROMACS, LAMMPS, and NAMD (private) which are integration tested against the IMDClient in the imdclient repo:

• GROMACS
• LAMMPS

Describe the solution you'd like

MDAnalysis adopts the IMDReader, brings on imdclient as an optional dependency

I propose that the core MDAnalysis codebase adopt the IMDReader as a trajectory reader with imdclient (which contains the IMDClient) as an optional dependency for MDAnalysis exactly in the same way h5py is an optional dependency that allows using the H5MDReader

The IMDClient handles the protocol communication at a binary level, wrapping it with a simple API:

• get_imdframe(): returns a singular IMDFrame (similar to a Timestep) of data from the simulation engine
• get_imdsessioninfo(): returns the data the user can expect in each IMDFrame
• stop(): ensures socket cleanup occurs

The IMDReader is just essentially a wrapper/adapter for the IMDClient which makes it compatible with the MDAnalysis reader API.

This way of structuring the code allows offloading the complicated simulation engine integration tests (which include docker containers and take quite a while to run) to the imdclient MDAKit dependency and allows other people outside of MDAnalysis to use the IMDv3 protocol without needing the IMDReader.

Currently, both the IMDReader and IMDClient are in the imdclient MDAKit for development purposes.

Protocol + integration tests go in the imdclient MDAKit, but reader API tests go in MDAnalysis

I also propose splitting up the tests into lightweight reader api tests that work with a "dummy" simulation engine (https://github.com/Becksteinlab/imdclient/blob/main/imdclient/tests/test_imdreader.py) and the more complicated protocol and simulation engine integration tests (protocol https://github.com/Becksteinlab/imdclient/blob/main/imdclient/tests/test_imdclient.py) (integration https://github.com/Becksteinlab/imdclient/blob/main/imdclient/tests/base.py).

MDAnalysis can adopt the lightweight tests that are relevant to MDAnalysis, and the imdclient MDAKit can keep everything else.

Summary

Things that move to MDAnalysis:

• Stream reader base class, which makes MDAnalysis broadly compatible with one-way streams
• IMDReader, which inherits from the Stream Reader base class
• IMDReader tests, which use a dummy server to allow running the standard Reader API tests.

Things that stay in imdclient, the proposed dependency:

• IMDClient, which handles the protocol
• All other heavyweight testing

Describe alternatives you've considered

Keep the IMDReader in the imdclient repository, never adopting it in the main codebase.