Skip to content

Allow listing outside URLs in extras #2103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Allow listing outside URLs in extras #2103

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
639afcc
Allow listing outside URLs in extras
sorentwo Mar 31, 2025
7d56189
Omit url extras when generating html paths
sorentwo Apr 7, 2025
c18894d
Skip generating files for extras in epub formatter
sorentwo Apr 7, 2025
b873af5
Prune unnecessary map fields for url extras
sorentwo Apr 7, 2025
a9be4ab
Document the new :url option in mix docs
sorentwo Apr 7, 2025
861d451
Merge branch 'main' into allow-extra-urls
sorentwo Apr 7, 2025
6b6cad6
Expand extra matcher to handle grouping urls
sorentwo Apr 15, 2025
d6e824b
Correct filtering logic for epub extras
sorentwo Apr 15, 2025
b5b224c
Update remixicon and add external-link icon
sorentwo Apr 15, 2025
b973cfc
Omit external links from search data
sorentwo Apr 15, 2025
64d85dd
Split customizing extras section apart
sorentwo Apr 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion assets/js/sidebar/sidebar-list.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ export function initialize () {
const items = []
const hasHeaders = Array.isArray(node.headers)
const translate = hasHeaders ? undefined : 'no'
const href = node?.url || `${node.id}.html`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add an icon, such as this one, for URLs?

If so, you can upload this bundle to remixicon.com, add external link, and get the new font back: https://github.com/elixir-lang/ex_doc/blob/main/assets/fonts/RemixIconCollection.remixicon

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2025-04-15 at 15 02 26

How is that?


// Group header.
if (node.group !== group) {
Expand All @@ -78,7 +79,7 @@ export function initialize () {
}

items.push(el('li', {}, [
el('a', {href: `${node.id}.html`, translate}, [node.nested_title || node.title]),
el('a', {href: href, translate}, [node.nested_title || node.title]),
...childList(`node-${node.id}-headers`,
hasHeaders
? renderHeaders(node)
Expand Down
8 changes: 4 additions & 4 deletions formatters/html/dist/html-5PS32TPG.js → formatters/html/dist/html-AG646WP7.js
View file
Open in desktop

Large diffs are not rendered by default.

37 changes: 32 additions & 5 deletions lib/ex_doc/formatter/html.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,7 @@ defmodule ExDoc.Formatter.HTML do
defp generate_extras(extras, config) do
generated_extras =
extras
|> Enum.reject(&is_map_key(&1, :url))
|> with_prev_next()
|> Enum.map(fn {node, prev, next} ->
filename = "#{node.id}.html"
Expand Down Expand Up @@ -349,6 +350,7 @@ defmodule ExDoc.Formatter.HTML do

extras =
config.extras
|> Enum.map(&normalize_extras/1)
|> Task.async_stream(
&build_extra(&1, groups, language, autolink_opts, source_url_pattern),
timeout: :infinity
Expand Down Expand Up @@ -384,10 +386,31 @@ defmodule ExDoc.Formatter.HTML do
end)
end

defp normalize_extras(base) when is_binary(base), do: {base, %{}}
defp normalize_extras({base, opts}), do: {base, Map.new(opts)}

defp disambiguate_id(extra, discriminator) do
Map.put(extra, :id, "#{extra.id}-#{discriminator}")
end

defp build_extra({input, %{url: _} = input_options}, groups, _lang, _auto, _url_pattern) do
input = to_string(input)
title = input_options[:title] || filename_to_title(input)
group = GroupMatcher.match_extra(groups, input)

# TODO: Can we make the content/source a link?

%{
group: group,
id: Utils.text_to_id(title),
source_path: input_options[:url],
source_url: input_options[:url],
title: title,
title_content: title,
url: input_options[:url]
}
end

defp build_extra({input, input_options}, groups, language, autolink_opts, source_url_pattern) do
input = to_string(input)
id = input_options[:filename] || input |> filename_to_title() |> Utils.text_to_id()
Expand Down Expand Up @@ -447,10 +470,6 @@ defmodule ExDoc.Formatter.HTML do
}
end

defp build_extra(input, groups, language, autolink_opts, source_url_pattern) do
build_extra({input, []}, groups, language, autolink_opts, source_url_pattern)
end

defp normalize_search_data!(nil), do: nil

defp normalize_search_data!(search_data) when is_list(search_data) do
Expand Down Expand Up @@ -602,7 +621,15 @@ defmodule ExDoc.Formatter.HTML do

{path, opts} ->
base = path |> to_string() |> Path.basename()
{base, opts[:filename] || Utils.text_to_id(Path.rootname(base))}

txid =
cond do
filename = opts[:filename] -> filename
url = opts[:url] -> url
true -> Utils.text_to_id(Path.rootname(base))
end

{base, txid}
end)
end
end
60 changes: 30 additions & 30 deletions lib/ex_doc/formatter/html/search_data.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,36 +18,8 @@ defmodule ExDoc.Formatter.HTML.SearchData do
["searchData=" | ExDoc.Utils.to_json(data)]
end

defp extra(map) do
if custom_search_data = map[:search_data] do
extra_search_data(map, custom_search_data)
else
Comment on lines -22 to -24
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a clause for this rather than a conditional in the function body.

{intro, sections} = extract_sections_from_markdown(map.source)

intro_json_item =
encode(
"#{map.id}.html",
map.title,
:extras,
intro
)

section_json_items =
for {header, body} <- sections do
encode(
"#{map.id}.html##{Utils.text_to_id(header)}",
header <> " - #{map.title}",
:extras,
body
)
end

[intro_json_item | section_json_items]
end
end

defp extra_search_data(map, custom_search_data) do
Enum.map(custom_search_data, fn item ->
defp extra(%{search_data: search_data} = map) when is_list(search_data) do
Enum.map(search_data, fn item ->
link =
if item.anchor === "" do
"#{map.id}.html"
Expand All @@ -59,6 +31,34 @@ defmodule ExDoc.Formatter.HTML.SearchData do
end)
end

defp extra(%{url: url} = map) do
[encode("#{map.id}", map.title, :extras, url)]
end

defp extra(map) do
{intro, sections} = extract_sections_from_markdown(map.source)

intro_json_item =
encode(
"#{map.id}.html",
map.title,
:extras,
intro
)

section_json_items =
for {header, body} <- sections do
encode(
"#{map.id}.html##{Utils.text_to_id(header)}",
header <> " - #{map.title}",
:extras,
body
)
end

[intro_json_item | section_json_items]
end

defp module(%ExDoc.ModuleNode{} = node) do
{intro, sections} = extract_sections(node.doc_format, node)

Expand Down
19 changes: 9 additions & 10 deletions lib/ex_doc/formatter/html/templates.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,9 @@ defmodule ExDoc.Formatter.HTML.Templates do

defp sidebar_extras(extras) do
for extra <- extras do
%{id: id, title: title, group: group, content: content} = extra
%{id: id, title: title, group: group} = extra

item =
%{
id: to_string(id),
title: to_string(title),
group: to_string(group),
headers: extract_headers(content)
}
item = %{id: to_string(id), title: to_string(title), group: to_string(group)}

case extra do
%{search_data: search_data} when is_list(search_data) ->
Expand All @@ -103,10 +97,15 @@ defmodule ExDoc.Formatter.HTML.Templates do
}
end)

Map.put(item, :searchData, search_data)
item
|> Map.put(:headers, extract_headers(extra.content))
|> Map.put(:searchData, search_data)

%{url: url} when is_binary(url) ->
Map.put(item, :url, url)

_ ->
item
Map.put(item, :headers, extract_headers(extra.content))
end
end
end
Expand Down
2 changes: 1 addition & 1 deletion mix.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
%{
"earmark_parser": {:hex, :earmark_parser, "1.4.43", "34b2f401fe473080e39ff2b90feb8ddfeef7639f8ee0bbf71bb41911831d77c5", [:mix], [], "hexpm", "970a3cd19503f5e8e527a190662be2cee5d98eed1ff72ed9b3d1a3d466692de8"},
"earmark_parser": {:hex, :earmark_parser, "1.4.44", "f20830dd6b5c77afe2b063777ddbbff09f9759396500cdbe7523efd58d7a339c", [:mix], [], "hexpm", "4778ac752b4701a5599215f7030989c989ffdc4f6df457c5f36938cc2d2a2750"},
"easyhtml": {:hex, :easyhtml, "0.3.2", "050adfc8074f53b261f7dfe83303d864f1fbf5988245b369f8fdff1bf4c4b3e6", [:mix], [{:floki, "~> 0.35", [hex: :floki, repo: "hexpm", optional: false]}], "hexpm", "b6a936f91612a4870aa3e828cd8da5a08d9e3b6221b4d3012b6ec70b87845d06"},
"floki": {:hex, :floki, "0.36.2", "a7da0193538c93f937714a6704369711998a51a6164a222d710ebd54020aa7a3", [:mix], [], "hexpm", "a8766c0bc92f074e5cb36c4f9961982eda84c5d2b8e979ca67f5c268ec8ed580"},
"jason": {:hex, :jason, "1.4.4", "b9226785a9aa77b6857ca22832cffa5d5011a667207eb2a0ad56adb5db443b8a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "c5eb0cab91f094599f94d55bc63409236a8ec69a21a67814529e8d5f6cc90b3b"},
Expand Down
18 changes: 18 additions & 0 deletions test/ex_doc/formatter/html_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,24 @@ defmodule ExDoc.Formatter.HTMLTest do
assert to_string(bar_content["h1"]) == "README bar"
end

test "extras defined as external urls", %{tmp_dir: tmp_dir} = context do
config =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add a test that shows grouping work. Imagine you want all URLs in a section in the sidebar called Important Links. The Template.sidebar_extras kinda implies URLs work with groups but we need to be sure it works "end to end".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems groups expect the filename but there is none here, so we probably have to pass the filename or the url (and update its docs accordingly).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grouping does work based on the input name, but it doesn't consider the URL. For example, with the extra [Elixir: [url: "https://elixir-lang.org"]] this will match on "Elixir", but it doesn't consider the url value at all.

    input = to_string(input)
    title = input_options[:title] || input
    group = GroupMatcher.match_extra(groups, input)

That's a little different than how regular grouping works because the input lacks path information for easy grouping.

I've opted to expand GroupMatcher.match_extra to be url/path aware, so the pattern applies to either the title or url.

doc_config(context,
extras: [
"README.md",
"Elixir": [url: "https://elixir-lang.org"]
]
)

File.write!("#{tmp_dir}/README.md", "README")

generate_docs(config)

content = File.read!(tmp_dir <> "/html/README.html")

assert content =~ "https://elixir-lang.org"
end

test "warns when generating an index.html file with an invalid redirect",
%{tmp_dir: tmp_dir} = context do
output =
Expand Down