Update EEP-74 to reflect non-centralized proposal #79
Conversation
This commit changes the error index to be non-centralized and also irons out a lot of the details on how it is expected to be used with various tools.
garazdawi
changed the title
| The namespaces for diagnostic indexes are not maintained centrally, but | ||
| left up to the community to co-ordinate. It is recommended that one | ||
| searches online before taking a namespace, just as when deciding on an | ||
| application name. |
There was a problem hiding this comment.
You appear to claim Ericsson should determine all namespaces used with Erlang errors, is that correct? Why not add a namespace concept to the Erlang programming language, to avoid a central naming bottleneck?
There was a problem hiding this comment.
You appear to claim Ericsson should determine all namespaces used with Erlang errors, is that correct?
No it is not correct. The intention was to claim the exact opposite, that we should NOT determine any namespace. I think that is quite clear from the text, what made you think that was not so?
There was a problem hiding this comment.
Later on in the document namespaces are defined as examples for Erlang/OTP source code that creates an association between the namespace name and the application name like ERL == compiler as shown in the output of application:get_diagnostic("ERL-0001"). The situation would be a lot simpler if we could just use application names as namespaces. Why add a separate name as a namespace that still must be associated with an application name?
There was a problem hiding this comment.
In the original proposal the error index namespace was not tied to an application, so I continued with using the new namespace. I did considered using the application name for the namespace, but as I don't expect there to be a lot of tools producing diagnostic codes I decided to decouple them in order to allow using a shorter prefix for the diagnostic code.
It is of course allowed for a tool to use the application name as the namespace.
There was a problem hiding this comment.
If the namespace name lacks visibility, it only makes the diagnostic code more obscure and difficult to utilize (e.g., to discover its context with search engine results). I understand the goal of having a shorter prefix for diagnostic codes but see it as creating new problems, instead of improving diagnostic codes. The new problems are 1) how to avoid conflicts with all Erlang source code everywhere, 2) how to associate the diagnostic code prefix with an Erlang/OTP application name that relates to the release process, any repository storage, and its existence on the filesystem (the directory path could contain the application name, the application file name and its contents, and the application's module name prefixes). Erlang/OTP application names are typically short and easy to remember, so they provide a good reference to find any mention of that source code and documentation.
Using the word "namespace" is also a problem because I was immediately wanting an application name module prefix. I understand that idea has been avoided for decades and the old package idea in Erlang was removed because Erlang/OTP systems are generally kept small to avoid any concerns regarding namespaces and multiple versions of the same dependencies.
Since it is difficult to anticipate use it should be best to make diagnostic codes easy to reference and that should require the use of application names for diagnostic code prefixes, because all Erlang source code is typically referenced based on an application name.
There was a problem hiding this comment.
Yes you need to have the application in the path and you need to search for the correct application, this is what I mean by "searching". You do not need either with a URI scheme, you just need access to the registry, and you can embed that registry much easier than having to include all the tools.
Think of a use case where you'd check for errors coming from CI logs for example. In your case you need to have all applications that produce error codes in your path. With my proposal you can just have a small escript do the job.
You could add a registry to your proposal as well as parsing rules for the error codes and whatnot, but you'd just be reinventing the wheel and potentially making the diagnostic class and error format a lot stricter than they should be. If I was using this in Cowboy it would make a lot more sense if I could have something like cow/<proto>/<code> for example, possibly with the <code> part being a word or two considering the number of possible errors. You could still provide recommendations for keeping them short without requiring it.
There was a problem hiding this comment.
you just need access to the registry
I think the word "just" here is doing a lot of work. In my proposal you just need access to the application that generated the diagnostic.
In my opinion we should strive to avoid a centralized registry of any kind, especially one that we embed into Erlang/OTP. If we do end up wanting a central registry, then hex is the closest thing we have in the Erlang world and then we are back at @okeuday's reasoning that we should use application names in the diagnostic codes to make translation trivial. We could work with the hex team to allow redirects to happen from https://hex.pm/packages/cowboy/diagnostics/* to https://ninenines.eu/docs/en/cowboy/2.13/manual/*, or just accept that applications without documention on hexdocs.pm will not get URIs.
As an aside, the idea of the documentation_url comes from a request of ExDoc users to be able to use autolinks to documentation not on hexdocs.pm. I'm just piggybacking on that idea.
Think of a use case where you'd check for errors coming from CI logs for example.
If you are parsing the output then you should probably want the json output from the EEP and in there you have the URL already. But there are bound to be some scenarious when you cannot use json and then the you may have to resort to using your favourite search engine or LLM.
There was a problem hiding this comment.
A registry is just a table of data. The only centralized part about it is that there has to be a single source of truth for it. Beyond that, it's just data.
A script could have the data from the registry embedded directly in the code, or have the parts of the registry it cares about handled as code directly. It doesn't have to ship with OTP in its entirety, although it could if useful. In practice I suppose OTP would only include the data it needs from the registry.
The EEPs are a kind of centralized registry that OTP team manages. The diagnostics class registry would be similar, only much simpler as each entry only needs a small number of fields. Here's an example of registry: https://www.iana.org/assignments/http-fields/http-fields.xhtml - a table. For diagnostics the fields would likely be the diagnostic class e.g erlc which must be unique, the application or local URI for local lookup, the web URI for online viewing and perhaps some one sentence description. You can manage the registry as a single text file with changes done via PRs, you don't really need any more than that. Could be an EEP that gets updated as necessary. Hex is very far from this idea.
Regardless of the solution you go for you will need to ensure that the diagnostic class is unique, and a registry helps with that.
There was a problem hiding this comment.
I just realized that one reason for not using a registry is because it would make it impossible to do a diagnostic code to URL mapping for private/proprietary projects.
It would also be possible to scrape the projects on hex.pm and create a hex application for that registry to make it available to everyone (except Erlang/OTP that is).
Regardless of the solution you go for you will need to ensure that the diagnostic class is unique, and a registry helps with that.
Why would we need that? We don't have any such mechanism for many things in Erlang (applications, modules, process names etc etc) and it has worked ok for now? From time to time there have been collisions, but any collisions that would take place can be solved by the people building the applications involved. I imagine the same would happen here.
There was a problem hiding this comment.
A registry doesn't prevent private projects to do diagnostic code to URL mapping, a registry is just data and private projects can provide their own data for the mapping. Just like there are application-specific "proprietary" HTTP headers that aren't in the HTTP header registry (GRPC has some for example). In our case, tools that read diagnostics just need a way to be fed the private data.
Proprietary projects can register their own diagnostic class and mappings even if fewer people would encounter them. I would imagine diagnostics to be useful to QuickCheck for example, and can't think of a reason to prevent them from registering.
It would also be possible to scrape the projects on hex.pm
It is much simpler to maintain a text file with the occasional PR. There aren't that many projects that would truly benefit from diagnostics. Even its use in Cowboy would be a stretch, I'm only considering it because I have things like this all over the place: https://github.com/ninenines/cowlib/blob/master/src/cow_http2_machine.erl#L381-L383 which could be replaced by diagnostics like cow/h2/data_when_idle. But there aren't too many non-CLI projects that have this amount of error details around.
Why would we need that?
That one was a general "you", not OTP team. Someone creating a new diagnostic class needs to make sure there's no conflicts, and a registry is a good start to do that.
Thanks for digging into the idea in any case!
|
We are merging the current version for now. There will probably be updates to this document before the final implementation is in place. |
This commit changes the error index to be non-centralized and also irons out a lot of the details on how it is expected to be used with various tools.
/cc @robertoaloi