Skip to content

Interop Type Mapping #110691

Open
@AaronRobinsonMSFT

Description

@AaronRobinsonMSFT

Interop Type Mapping

Work items:

Background

When interop between languages/platforms involves the projection of types, some kind of type mapping logic must often exist. This mapping mechanism is used to determine what .NET type should be used to project a type from language X and vice versa.

The most common mechanism for this is the generation of a large look-up table at build time, which is then injected into the application or Assembly. If injected into the Assembly, there is typically some registration mechanism for the mapping data. Additional modifications and optimizations can be applied based on the user experience or scenarios constraints (that is, build time, execution environment limitations, etc).

At present, there are at least three (3) bespoke mechanisms for this in the .NET ecosystem:

Related issue(s):

Proposal

The .NET ecosystem should provide an official API and process for handling type mapping in interop scenarios.

Priorties

  1. Trimmer friendly - AOT compatible.
  2. Usable from both managed and unmanaged environments.
  3. Low impact to application start-up and/or Assembly load.
  4. Be composable - handle multiple type mappings.

The below .NET APIs represents only part of the feature. The complete scenario would involve additional steps and tooling.

Provided by BCL (that is, NetCoreApp)

namespace System.Runtime.InteropServices;

/// <summary>
/// Base interface for target type universe.
/// </summary>
/// <remarks>
/// This interface should be inherited directly by a concrete type
/// universe class. APIs consuming <see cref="ITypeMapUniverse"/> as
/// generic parameters don't respect class inheritance.
/// </remarks>
public interface ITypeMapUniverse { }

/// <summary>
/// Type mapping between a string and a type.
/// </summary>
/// <typeparam name="TTypeUniverse">Type universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = true)]
public sealed class TypeMapAttribute<TTypeUniverse> : Attribute
    where TTypeUniverse : ITypeMapUniverse
{
    /// <summary>
    /// Create a mapping between a value and a <see cref="System.Type"/>.
    /// </summary>
    /// <param name="value">String representation of key</param>
    /// <param name="target">Type value</param>
    /// <remarks>
    /// This mapping is unconditionally inserted into the type map.
    /// </remarks>
    public TypeMapAttribute(string value, Type target)
    { }

    /// <summary>
    /// Create a mapping between a value and a <see cref="System.Type"/>.
    /// </summary>
    /// <param name="value">String representation of key</param>
    /// <param name="target">Type value</param>
    /// <param name="trimTarget">Type used by Trimmer to determine type map inclusion.</param>
    /// <remarks>
    /// This mapping is only included in the type map if the Trimmer observes a type check
    /// using the <see cref="System.Type"/> represented by <paramref name="trimTarget"/>.
    /// </remarks>
    [RequiresUnreferencedCode("Interop types may be removed by trimming")]
    public TypeMapAttribute(string value, Type target, Type trimTarget)
    { }
}

/// <summary>
/// Declare an assembly that should be inspected during type map building.
/// </summary>
/// <typeparam name="TTypeUniverse">Type universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = true)]
public sealed class TypeMapAssemblyTargetAttribute<TTypeUniverse> : Attribute
    where TTypeUniverse : ITypeMapUniverse
{
    /// <summary>
    /// Provide the assembly to look for type mapping attributes.
    /// </summary>
    /// <param name="assemblyName">Assembly to reference</param>
    public TypeMapAssemblyTargetAttribute(string assemblyName)
    { }
}

/// <summary>
/// Create a type association between a type and its proxy.
/// </summary>
/// <typeparam name="TTypeUniverse">Type universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = false)]
public sealed class TypeMapAssociationAttribute<TTypeUniverse> : Attribute
    where TTypeUniverse : ITypeMapUniverse
{
    /// <summary>
    /// Create an association between two types in the type map.
    /// </summary>
    /// <param name="source">Target type.</param>
    /// <param name="proxy">Type to associated with <paramref name="source"/>.</param>
    /// <remarks>
    /// This mapping will only exist in the type map if the Trimmer observes
    /// an allocation using the <see cref="System.Type"/> represented by <paramref name="source"/>.
    /// </remarks>
    public TypeMapAssociationAttribute(Type source, Type proxy)
    { }
}

/// <summary>
/// Entry type for interop type mapping logic.
/// </summary>
public static class TypeMapping
{
    /// <summary>
    /// Returns the External type type map generated for the current application.
    /// </summary>
    /// <typeparam name="TTypeUniverse">Type universe</typeparam>
    /// <param name="map">Requested type map</param>
    /// <returns>True if the map is returned, otherwise false.</returns>
    /// <remarks>
    /// Call sites are treated as an intrinsic by the Trimmer and implemented inline.
    /// </remarks>
    [RequiresUnreferencedCode("Interop types may be removed by trimming")]
    public static bool TryGetExternalTypeMapping<TTypeUniverse>([NotNullWhen(returnValue: true)] out IReadOnlyDictionary<string, Type>? map)
        where TTypeUniverse : ITypeMapUniverse;

    /// <summary>
    /// Returns the associated type type map generated for the current application.
    /// </summary>
    /// <typeparam name="TTypeUniverse">Type universe</typeparam>
    /// <param name="map">Requested type map</param>
    /// <returns>True if the map is returned, otherwise false.</returns>
    /// <remarks>
    /// Call sites are treated as an intrinsic by the Trimmer and implemented inline.
    /// </remarks>
    [RequiresUnreferencedCode("Interop types may be removed by trimming")]
    public static bool TryGetTypeProxyMapping<TTypeUniverse>([NotNullWhen(returnValue: true)] out IReadOnlyDictionary<Type, Type>? map)
        where TTypeUniverse : ITypeMapUniverse;
}

Given the above types the following would take place.

  1. Types involved in unmanaged-to-managed interop operations would be referenced in a
    TypeMapAttribute assembly attribute that declared the external type system name, a target
    type, and optionally a "trim-target" type use by the Trimmer to determine if the target
    type should be included in the map. If the trim-target type is used in a type check, then
    the entry will be inserted into the map. If the TypeMapAttribute constructor that doesn't
    take a trim-target is used, the entry will be inserted unconditionally.

The target type would have interop specific "capabilities" (for example, create an instance).

  1. Types used in a managed-to-unmanaged interop operation would use TypeMapAssociationAttribute
    to define a conditional link between the source and proxy type. In other words, if the
    source is kept, so is the proxy type. If Trimmer observes an explicit allocation of the source
    type, the entry will be inserted into the map.

  2. During application build, source would be generated and injected into the application
    that defines appropriate TypeMapAssemblyTargetAttribute instances. This attribute would help the
    Trimmer know other assemblies to examine for TypeMapAttribute and TypeMapAssociationAttribute
    instances. These linked assemblies could also be used in the non-Trimmed scenario whereby we
    avoid creating the map at build-time and create a dynamic map at run-time instead.

  3. The Trimmer will build two maps based on the above attributes from the application reference
    closure.

    (a) Using TypeMapAttribute a map from string to target Type. If a trim-target Type
    was provided, the Trimmer will determine if it is used in a type check. If it was used in
    a type check, the mapping will be included. If the trim-target type is not provided, the mapping
    will be included unconditionally.

    (b) Using TypeMapAssociationAttribute a map from Type to Type (source to proxy).
    The map will only contain an entry if the Trimmer determines the source type was explicitly
    allocated.

    Note Conflicting key/value mappings in the same type universe would be reconciled by the
    application re-defining the mapping entry that will be in the respective map. If a conflict
    is still present, the build should fail.

    Note The emitted map format is a readonly binary blob that will be stored in the application
    assembly. The format of the binary blob is an implementation detail that will be passed to
    an internal type contained with CoreLib.

  4. The Trimmer will consider calls to TypeMapping.GetExternalTypeMapping<> and
    TypeMapping.GetTypeProxyMapping<> as intrinsic operations and replaced inline with the appropriate
    map instantiation (for example, Java via JavaTypeUniverse).

    IReadOnlyDictionary<string, Type> externalToType;
    TypeMapping.TryGetExternalTypeMapping<JavaTypeUniverse>(out externalToType);
    // The above will be replaced by the Trimmer with an instantiation of the appropriate map.
    // The "StaticTypeMap" being an internal type and an implementation detail provided by CoreLib.
    // Replaced with:
    //     IReadOnlyDictionary<string, Type> externalToType;
    //     externalToType = new StaticTypeMap(s_javaTypeUniverseMapBlob);

Example usage

Provided by .NET for Android runtime

// Type used to express specific type universe.
public sealed class JavaTypeUniverse : ITypeMapUniverse { }

// Java capability to enable type instance creation.
public interface IJavaCreateInstance
{
    object Create(ref JniObjectReference reference, JniObjectReferenceOptions options);
}

// Java capability implementation for object creation.
public class JavaObjectTypeMapAttribute<[DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] T> : Attribute, IJavaCreateInstance
    where T : JavaObject
{
    public object Create(ref JniObjectReference reference, JniObjectReferenceOptions options)
    {
        // Create instance using T.
    }
}

// Java capability implementation for string creation.
public class JavaStringTypeAttribute : Attribute, IJavaCreateInstance
{
    public object Create(ref JniObjectReference reference, JniObjectReferenceOptions options)
    {
        IntPtr ptr = /* Extract the underlying string pointer from reference. */
        string str = Marshal.PtrToStringUni(ptr);
        return str;
    }
}

// -- Alternative attribute design
// This would expose the direct as a Type instead of as a generic parameter. It would reduce
// type loading costs and could be the single attribute for all "capabilities".
public class JavaObjectTypeMapAttribute : Attribute
{
    public JavaObjectTypeMapAttribute([DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] Type type)
    { }

    public [DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] Type TargetType { get; }
}

.NET for Android projection library

// Dependent association between typeof(string) => typeof(StringProxy).
[assembly: TypeMapAssociation<JavaTypeUniverse>(typeof(string), typeof(StringProxy))]

// Map string name of Java type to typeof(StringProxy), using typeof(string) to indicate
// if the entry should be included or not.
[assembly: TypeMap<JavaTypeUniverse>("java/lang/String", typeof(StringProxy), typeof(string))]

[JavaStringType]
public class StringProxy
{ }

User application

// Application reference for assembly to be used in type map generation.
[assembly: TypeMapAssemblyTarget<JavaTypeUniverse>("Application.SideCar.dll")]

// User defined type
[JavaObjectTypeMap<MyJavaClass>]
public class MyJavaClass : JavaObject
{ }

var inst = new ClassFromNuGet();

Generated Application.SideCar.dll

A sidecar assembly could always be generated to handle the following:

  • Specify assembly type map targets (for example, .NET for Android runtime)
  • Specify type mappings for types defined in the application.
  • Generate proxies and mappings for types in Nugets to support older TFMs.
// Application reference for assembly to be used in type map generation.
[assembly: TypeMapAssemblyTarget<JavaTypeUniverse>("Android.Runtime.dll")]

[assembly: TypeMap<JavaTypeUniverse>("app/MyJavaClass", typeof(MyJavaClass), typeof(MyJavaClass))]

[assembly: TypeMap<JavaTypeUniverse>("lib/classFromNuget", typeof(ClassFromNuGetProxy), typeof(ClassFromNuGet))]

[JavaObjectTypeMap<ClassFromNuGet>]
internal class ClassFromNuGetProxy
{ }

Interop runtime (for example, .NET for Android) usage example.

// Stored by the interop runtime in some global static
static IReadOnlyDictionary<string, Type> s_JavaTypeMap;
TypeMapping.TryGetExternalTypeMapping<JavaTypeUniverse>(out s_JavaTypeMap);

...

string javaTypeName = /* Get Java type name from jniHandle */
s_JavaTypeMap.TryGetValue(javaTypeName, out Type? projection);
Debug.Assert(projection != null); // Assuming map contains type.

var creator = (IJavaCreateInstance)projection.GetCustomAttribute(typeof(JavaObjectTypeMapAttribute<>));
var objectRef = new JniObjectReference(jniHandle, JniObjectReferenceType.Local);
JavaObject mcw = (JavaObject)creator.Create(ref objectRef, JniObjectReferenceOptions.None);
Previous proposal
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct | AttributeTargets.Enum | AttributeTargets.Delegate | AttributeTargets.Interface, Inherited = false)]
public sealed class TypeMappingAttribute : Attribute
{
    public TypeMappingAttribute(string mapping);
}

The above attribute would be searched in all assemblies passed to a tool/MSBuild Task/etc and the result would be a binary blob. This binary blob could be in one of several forms (a) a .cs file that defines a static ReadOnlySpan<byte>, (b) a binary file that could be embedded in a .NET assembly as a resource, or perhaps another option.

Question Are there security concerns with binary blobs being written to disk?

Question: The trimmer must run prior to binary blob generation since it wouldn't be trimmable. How does this impact the workflow?

Type mapping scenarios and trimmer:

Dynamic: No special tooling has run over the whole app. The type mappings are per-assembly and registered at runtime. Tooling can be used to generate or optimize the per-assembly mapping. This should handle plugins where a new assembly with additional mappings shows up in flight.

IL trimming: The IL trimmer should not treat the types involved in the mapping as roots - if the type is not otherwise referenced by the app, it should be removed from the mapping. IL trimmer (or some other tool) generates the per-app mapping blob that is consumed at runtime. It may be easiest to make this a IL trimmer feature. This scenario does not handle plugins.

AOT: It is similar to IL trimming case, but the exact format of the blob may need to be different - both to make it more efficient and to avoid dependency on metadata tokens that expect the IL trimming implementation is going to have.

This API would be based on the generated static data of type mappings being generated using a static hashing approach for the data. From the .NET side, it could be implemented through FrozenDictionary<TKey, TValue> and an instantiated from the generated data. A static ReadOnlySpan<byte> field inserted into the application or Assembly would integration seamless, but an embedded resource is also workable. This concept is similar to existing technologies in C, such as gperf or C Minimal Perfect Hashing Library. Size and performance should be measured using different approaches.

The API shape would be a mapping from a TKey, likely a string, to an internal type, conceptually containing the following:

public struct TypeMapValue
{
    public TBD LookUpData;
    public int Index;
}

The above type contains a field for the discovery of the type, not presently loaded, and the second represents an index into an array that contains the loaded type to use. The strawman example below helps illustrate the workflow.

TypeMapping<RuntimeTypeHandle> s_XtoNETMap = new (
    s_generated_XToNET,
    (TBD data) =>
    {
        // User loads System.Type based on data.
        return type.TypeHandle;
    });

TypeMapping<IntPtr> s_NETToXMap = new (
    s_generated_NETToX,
    (TBD data) =>
    {
        // User loads type in X based on data.
    });

public sealed unsafe class TypeMapping<TType>
    where TType : unmanaged
{
    private readonly FrozenDictionary<Key, TypeMapValue> _lookUp;
    private readonly TType[] _loadedType;
    private readonly delegate* <ReadOnlySpan<byte>, Key> _hashFunction;
    private readonly Func<TBD, TType> _typeLoad;

    public TypeMapping(ReadOnlySpan<byte> generatedTypeMap, Func<TBD, TType> typeLoad)
    {
        _lookUp = ParseData(generatedTypeMap, out HashFunction hash, out int count);
        _loadedType = new TType[count];
        _hashFunction = GetHashFunction(hash);
        _typeLoad = typeLoad;
    }

    public TType this[ReadOnlySpan<byte> key] => LookUp(key);

    private TType LookUp(ReadOnlySpan<byte> key)
    {
        // Verify the type is actually "known".
        if (!_lookUp.TryGetValue(_hashFunction(key), out TypeMapValue v))
            throw new Exception("Invalid type look-up");

        // Check if the type is loaded.
        ref TType tgt = ref _loadedType[v.Index];

        // Defer to the user if the type isn't loaded.
        if (default(TType).Equals(tgt))
            tgt = _typeLoad(v.LookUpData);

        return tgt;
    }
}

Metadata

Metadata

Assignees

No one assigned

    Projects

    Status

    No status

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions