I have an inserting story with a not less of an interesting question.
So. I am currently making a tool for my company that parses data received from MEX endpoint of a WCF service. Everything is okay about that, but i was requested to compile my project to native AOT to exclude a need of installing runtime and increase performance yada-yada. An absolutely common request.
But, in testing, testers have started getting this error:
System.InvalidOperationException
HResult=0x80131509
Message = There is an error in XML document (0, 0).
Source = System.Private.Xml
Stack trace:
in System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
in System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader)
in System.Web.Services.Description.ServiceDescription.Read(XmlReader reader, Boolean validate)
in System.Web.Services.Description.ServiceDescription.Read(Stream stream, Boolean validate)
in DocSynth.Remote.WCF.Services.WSDLParser.ReadWSDLFiles(IEnumerable`1 filePaths)
in DocSynth.Remote.WCF.Services.WCFMEXMetadataProvider.ParseWCFFiles(WCFFilesPreparationResult files)
Initial Stack trace:
System.ArgumentNullException.Throw(string)
System.ArgumentNullException.ThrowIfNull(object, string)
System.Xml.Serialization.XmlReflectionImporter.ImportTypeMapping(System.Type, System.Xml.Serialization.XmlRootAttribute, string)
System.Xml.Serialization.XmlSerializer.GetMapping()
System.Xml.Serialization.XmlSerializer.Deserialize(System.Xml.XmlReader, string, System.Xml.Serialization.XmlDeserializationEvents)
Internal Exception 1:
ArgumentNullException: Value cannot be null. Arg_ParamName_Name
What? What error? at (0, 0)?
I knew for sure that .WSDL file wasn't at fault, because it came from MEX endpoint, and I've checked it manually. Nothing special that screamed "XML error" to me.
So I've started exploring: From this MSDN post I've learned that when XMLTextReader is created with a Stream used, it advances the stream by 4096. And that person solved his problem by setting fs.Position = 0;. Tried that, didn't help.
Next: I've started checking out the code piece by piece. When i called ServiceDescription.Read(fs) it have executed the following code:
XMLSerializer.cs
[RequiresUnreferencedCode(TrimDeserializationWarning)]
public object? Deserialize(XmlReader xmlReader, string? encodingStyle, XmlDeserializationEvents events)
{
events.sender = this;
try
{
if (_primitiveType != null)
{
if (encodingStyle != null && encodingStyle.Length > 0)
{
throw new InvalidOperationException(SR.Format(SR.XmlInvalidEncodingNotEncoded1, encodingStyle));
}
return DeserializePrimitive(xmlReader, events);
}
// This 'else if' was executed
else if (ShouldUseReflectionBasedSerialization(_mapping) || _isReflectionBasedSerializer)
{
return DeserializeUsingReflection(xmlReader, encodingStyle, events);
}
else if (_tempAssembly == null || _typedSerializer)
{
XmlSerializationReader reader = CreateReader();
reader.Init(xmlReader, events, encodingStyle);
return Deserialize(reader);
}
else
{
return _tempAssembly.InvokeReader(_mapping, xmlReader, events, encodingStyle);
}
}
...
...
...
Okay... it tried to deserialize using reflection, and internally fell because '_rootType' was null inside function GetMapping...
XMLSerializer.cs
[RequiresUnreferencedCode("calls GenerateXmlTypeMapping")]
private XmlMapping GetMapping()
{
if (_mapping == null || !_mapping.GenerateSerializer)
{
_mapping = GenerateXmlTypeMapping(_rootType!, null, null, null, DefaultNamespace);
}
return _mapping;
}
Okay then... why is the _rootType null? I'm working with a specific serializer that has context about what XML file it is about to encounter. So I've checked what actually is inside ServiceDesction's serializer:
ServiceDescription.cs
internal class ServiceDescriptionSerializer : XmlSerializer
{
protected override XmlSerializationReader CreateReader()
{
return new ServiceDescriptionSerializationReader();
}
protected override XmlSerializationWriter CreateWriter()
{
return new ServiceDescriptionSerializationWriter();
}
public override bool CanDeserialize(System.Xml.XmlReader xmlReader)
{
return xmlReader.IsStartElement("definitions", Namespace);
}
protected override void Serialize(Object objectToSerialize, XmlSerializationWriter writer)
{
((ServiceDescriptionSerializationWriter)writer).Write125_definitions(objectToSerialize);
}
protected override object Deserialize(XmlSerializationReader reader)
{
return ((ServiceDescriptionSerializationReader)reader).Read125_definitions();
}
}
[XmlIgnore]
public static XmlSerializer Serializer
{
get
{
if (s_serializer == null)
{
WebServicesSection config = WebServicesSection.Current;
XmlAttributeOverrides overrides = new XmlAttributeOverrides();
XmlSerializerNamespaces ns = new XmlSerializerNamespaces();
ns.Add("s", XmlSchema.Namespace);
WebServicesSection.LoadXmlFormatExtensions(config.GetAllFormatExtensionTypes(), overrides, ns);
s_namespaces = ns;
s_serializer = new ServiceDescriptionSerializer();
s_serializer.UnknownElement += new XmlElementEventHandler(RuntimeUtils.OnUnknownElement);
}
return s_serializer;
}
}
First thing I've noticed, that this file in dotnet/wcf repository was last changed 4 years ago so no way that I'm using a wrong library version, so then why did it started breaking and telling me this error even for files I've tried and parsed while developing main features?
I've started looking at this serialized and... okay, they're creating a new instance of ServiceDescriptionSerializer which is defined right above this getter. Okay... That ServiceDescriptionSerializer class is just a custom wrapper around XmlSerializer that used custom reader and writer. Oh that's why '_rootType' is 'null'! It doesn't require a root type, because custom reader internally will handle everything.
But wait... why does it use reflection then? It has custom reader, so it should fall into last else if that actually Calls overridden CreateReader(). I've started investigating ShouldUseReflectionBasedSerialization(_mapping) method:
XmlSerializer.cs
private static bool ShouldUseReflectionBasedSerialization(XmlMapping mapping)
{
return Mode == SerializationMode.ReflectionOnly
|| (mapping != null && mapping.IsSoap);
}
The _mapping is null, because it is never created or populated. The constructor of XmlSerializer used is empty. So second clause is always true. So the overall value depends only on Mode is was ReflectionOnly and I went to check what this Mode is, and why it's value is ReflectionOnly:
XmlSerializer.cs
internal static SerializationMode Mode
{
get => RuntimeFeature.IsDynamicCodeSupported ? s_mode : SerializationMode.ReflectionOnly;
set => s_mode = value;
}
And it clicked! 2 weeks ago I've enabled a checkbox in project properties "Enable native AOT publication" so of course RuntimeFeature.IsDynamicCodeSupported is false. Of course that ternary operator return ReflectionOnly. I've changed that checkbox, reported that we will have to create our own WSDL parser if we want to use this tool compiled natively.
But then frustration came, I've spent solid 1.5 weeks on this bug, because this is literally the first ever occasion of this error, I've googled thoroughly and no one ever had this bug. I hope this page gets indexated by search engines and if someone would encounter this bug, it'll help them.
But the question still stands. Why? why does XmlSerializer requires dynamic code generation for serializing if the program is built in native code? It seems very strange, when compiling native code, reflection shouldn't be really used to edit or create types at runtime, all of the types should be compiled and packaged inside executable, no?
And the super generic use case for XmlSerializer is like this:
XmlSerialier serializer = new XmlSerializer(...) { ... };
...
Foo fooObject = (Foo)serializer.Deserialize(reader);
All the type and fields information is present and is compiled, so why?
why does XmlSerializer requires dynamic code generation for deserialization with custom reader??
I have an inserting story with a not less of an interesting question.
So. I am currently making a tool for my company that parses data received from MEX endpoint of a WCF service. Everything is okay about that, but i was requested to compile my project to native AOT to exclude a need of installing runtime and increase performance yada-yada. An absolutely common request.
But, in testing, testers have started getting this error:
System.InvalidOperationException
HResult=0x80131509
Message = There is an error in XML document (0, 0).
Source = System.Private.Xml
Stack trace:
in System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
in System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader)
in System.Web.Services.Description.ServiceDescription.Read(XmlReader reader, Boolean validate)
in System.Web.Services.Description.ServiceDescription.Read(Stream stream, Boolean validate)
in DocSynth.Remote.WCF.Services.WSDLParser.ReadWSDLFiles(IEnumerable`1 filePaths)
in DocSynth.Remote.WCF.Services.WCFMEXMetadataProvider.ParseWCFFiles(WCFFilesPreparationResult files)
Initial Stack trace:
System.ArgumentNullException.Throw(string)
System.ArgumentNullException.ThrowIfNull(object, string)
System.Xml.Serialization.XmlReflectionImporter.ImportTypeMapping(System.Type, System.Xml.Serialization.XmlRootAttribute, string)
System.Xml.Serialization.XmlSerializer.GetMapping()
System.Xml.Serialization.XmlSerializer.Deserialize(System.Xml.XmlReader, string, System.Xml.Serialization.XmlDeserializationEvents)
Internal Exception 1:
ArgumentNullException: Value cannot be null. Arg_ParamName_Name
What? What error? at (0, 0)?
I knew for sure that .WSDL file wasn't at fault, because it came from MEX endpoint, and I've checked it manually. Nothing special that screamed "XML error" to me.
So I've started exploring: From this MSDN post I've learned that when XMLTextReader is created with a Stream used, it advances the stream by 4096. And that person solved his problem by setting fs.Position = 0;. Tried that, didn't help.
Next: I've started checking out the code piece by piece. When i called ServiceDescription.Read(fs) it have executed the following code:
XMLSerializer.cs
[RequiresUnreferencedCode(TrimDeserializationWarning)]
public object? Deserialize(XmlReader xmlReader, string? encodingStyle, XmlDeserializationEvents events)
{
events.sender = this;
try
{
if (_primitiveType != null)
{
if (encodingStyle != null && encodingStyle.Length > 0)
{
throw new InvalidOperationException(SR.Format(SR.XmlInvalidEncodingNotEncoded1, encodingStyle));
}
return DeserializePrimitive(xmlReader, events);
}
// This 'else if' was executed
else if (ShouldUseReflectionBasedSerialization(_mapping) || _isReflectionBasedSerializer)
{
return DeserializeUsingReflection(xmlReader, encodingStyle, events);
}
else if (_tempAssembly == null || _typedSerializer)
{
XmlSerializationReader reader = CreateReader();
reader.Init(xmlReader, events, encodingStyle);
return Deserialize(reader);
}
else
{
return _tempAssembly.InvokeReader(_mapping, xmlReader, events, encodingStyle);
}
}
...
...
...
Okay... it tried to deserialize using reflection, and internally fell because '_rootType' was null inside function GetMapping...
XMLSerializer.cs
[RequiresUnreferencedCode("calls GenerateXmlTypeMapping")]
private XmlMapping GetMapping()
{
if (_mapping == null || !_mapping.GenerateSerializer)
{
_mapping = GenerateXmlTypeMapping(_rootType!, null, null, null, DefaultNamespace);
}
return _mapping;
}
Okay then... why is the _rootType null? I'm working with a specific serializer that has context about what XML file it is about to encounter. So I've checked what actually is inside ServiceDesction's serializer:
ServiceDescription.cs
internal class ServiceDescriptionSerializer : XmlSerializer
{
protected override XmlSerializationReader CreateReader()
{
return new ServiceDescriptionSerializationReader();
}
protected override XmlSerializationWriter CreateWriter()
{
return new ServiceDescriptionSerializationWriter();
}
public override bool CanDeserialize(System.Xml.XmlReader xmlReader)
{
return xmlReader.IsStartElement("definitions", Namespace);
}
protected override void Serialize(Object objectToSerialize, XmlSerializationWriter writer)
{
((ServiceDescriptionSerializationWriter)writer).Write125_definitions(objectToSerialize);
}
protected override object Deserialize(XmlSerializationReader reader)
{
return ((ServiceDescriptionSerializationReader)reader).Read125_definitions();
}
}
[XmlIgnore]
public static XmlSerializer Serializer
{
get
{
if (s_serializer == null)
{
WebServicesSection config = WebServicesSection.Current;
XmlAttributeOverrides overrides = new XmlAttributeOverrides();
XmlSerializerNamespaces ns = new XmlSerializerNamespaces();
ns.Add("s", XmlSchema.Namespace);
WebServicesSection.LoadXmlFormatExtensions(config.GetAllFormatExtensionTypes(), overrides, ns);
s_namespaces = ns;
s_serializer = new ServiceDescriptionSerializer();
s_serializer.UnknownElement += new XmlElementEventHandler(RuntimeUtils.OnUnknownElement);
}
return s_serializer;
}
}
First thing I've noticed, that this file in dotnet/wcf repository was last changed 4 years ago so no way that I'm using a wrong library version, so then why did it started breaking and telling me this error even for files I've tried and parsed while developing main features?
I've started looking at this serialized and... okay, they're creating a new instance of ServiceDescriptionSerializer which is defined right above this getter. Okay... That ServiceDescriptionSerializer class is just a custom wrapper around XmlSerializer that used custom reader and writer. Oh that's why '_rootType' is 'null'! It doesn't require a root type, because custom reader internally will handle everything.
But wait... why does it use reflection then? It has custom reader, so it should fall into last else if that actually Calls overridden CreateReader(). I've started investigating ShouldUseReflectionBasedSerialization(_mapping) method:
XmlSerializer.cs
private static bool ShouldUseReflectionBasedSerialization(XmlMapping mapping)
{
return Mode == SerializationMode.ReflectionOnly
|| (mapping != null && mapping.IsSoap);
}
The _mapping is null, because it is never created or populated. The constructor of XmlSerializer used is empty. So second clause is always true. So the overall value depends only on Mode is was ReflectionOnly and I went to check what this Mode is, and why it's value is ReflectionOnly:
XmlSerializer.cs
internal static SerializationMode Mode
{
get => RuntimeFeature.IsDynamicCodeSupported ? s_mode : SerializationMode.ReflectionOnly;
set => s_mode = value;
}
And it clicked! 2 weeks ago I've enabled a checkbox in project properties "Enable native AOT publication" so of course RuntimeFeature.IsDynamicCodeSupported is false. Of course that ternary operator return ReflectionOnly. I've changed that checkbox, reported that we will have to create our own WSDL parser if we want to use this tool compiled natively.
But then frustration came, I've spent solid 1.5 weeks on this bug, because this is literally the first ever occasion of this error, I've googled thoroughly and no one ever had this bug. I hope this page gets indexated by search engines and if someone would encounter this bug, it'll help them.
But the question still stands. Why? why does XmlSerializer requires dynamic code generation for serializing if the program is built in native code? It seems very strange, when compiling native code, reflection shouldn't be really used to edit or create types at runtime, all of the types should be compiled and packaged inside executable, no?
And the super generic use case for XmlSerializer is like this:
XmlSerialier serializer = new XmlSerializer(...) { ... };
...
Foo fooObject = (Foo)serializer.Deserialize(reader);
All the type and fields information is present and is compiled, so why?
why does XmlSerializer requires dynamic code generation for deserialization with custom reader??
Share Improve this question edited Feb 17 at 11:36 ZecosMAX asked Feb 17 at 11:21 ZecosMAXZecosMAX 2432 silver badges13 bronze badges 6 | Show 1 more comment1 Answer
Reset to default 2You're quite right that if you have a pre-compiled serializer then you could avoid generating dynamic serialization assemblies.
But you don't have those. All you have is AOT binaries, that is not enough for XmlSerializer. It expects you to pre-generate serialization assemblies using the XML tools. For .NET Framework that's the sgen.exe tool, and for .NET Core you need to add some lines to your project file before you build it.
<ItemGroup>
<DotNetCliToolReference Include="Microsoft.XmlSerializer.Generator" Version="8.0.0" />
</ItemGroup>
why does XmlSerializer requires dynamic code generation for serializing if the program is built in native code?Because AOT was introduced a long time AFTERXmlSerializer. People generally have a hard time supporting things that they don't know exist. – mjwills Commented Feb 17 at 22:41Deserializationis? It's the process where you take a file and assign appropriate fields within a known type provided by the program. Generics were around in C# since 2005 in C# 2.0 to provide type data. But they even require you to use reflection based deserializer if dynamic code is not supported (aka native build). My question is: why? why can i use only reflection based serializer if the build is native AOT? Program (in this case WCF lib) has provided custom serializer that has all the type context in the world – ZecosMAX Commented Feb 18 at 8:43XmlSerializerdevelopers had the ability to time travel. They didn't. They didn't know AOT was going to come in the future and break their code? How could they possibly know that? – mjwills Commented 2 days ago