Skip to content

Commit 069e7ba

Browse files
jmchiltonjmchilton
committed
Slides that live with the documentation!
 - Enable Markdown in the Sphinx docs. - Add a doc section for Galaxy tool framework updates. - Add a script to generate HTML Remark slides from a markdown document. - Use aforementioned script to auto-generate tool framework update slides from Planemo docs.
1 parent fea51fc commit 069e7ba

8 files changed

+589
-2
lines changed

‎dev-requirements.txt

+2-1
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,9 @@ tox
66
nose
77
coverage
88

9-
# For dev
9+
#Building Docs
1010
sphinx
11+
recommonmark
1112

1213
# Used to check readme.
1314
readme

‎docs/conf.py

+7-1
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,12 @@
3131
# version is used.
3232
sys.path.insert(0, project_root)
3333

34+
from recommonmark.parser import CommonMarkParser
35+
36+
source_parsers = {
37+
'.md': CommonMarkParser,
38+
}
39+
3440
import planemo
3541

3642
# -- General configuration ---------------------------------------------
@@ -46,7 +52,7 @@
4652
templates_path = ['_templates']
4753

4854
# The suffix of source filenames.
49-
source_suffix = '.rst'
55+
source_suffix = ['.rst', '.md']
5056

5157
# The encoding of source files.
5258
#source_encoding = 'utf-8-sig'

‎docs/galaxy_changelog.html

+276
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,276 @@
1+
<!DOCTYPE html>
2+
<html>
3+
<head>
4+
<title>tool_changes</title>
5+
<meta charset="utf-8">
6+
<style>
7+
@import url(https://fonts.googleapis.com/css?family=Yanone+Kaffeesatz);
8+
@import url(https://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic);
9+
@import url(https://fonts.googleapis.com/css?family=Ubuntu+Mono:400,700,400italic);
10+
11+
body { font-family: 'Droid Serif'; }
12+
h1, h2, h3 {
13+
font-family: 'Yanone Kaffeesatz';
14+
font-weight: normal;
15+
}
16+
.remark-code, .remark-inline-code { font-family: 'Ubuntu Mono'; }
17+
</style>
18+
</head>
19+
<body>
20+
<textarea id="source">
21+
22+
class: center, middle
23+
24+
## Galaxy Tool Framework Changes
25+
26+
This document describes changes to Galaxy's tooling framework over recent
27+
releases.
28+
29+
---
30+
31+
### 16.04
32+
33+
Full [Galaxy changelog](https://docs.galaxyproject.org/en/master/releases/16.04_announce.html).
34+
35+
---
36+
37+
#### Tool Profile Version ([PR #1688](https://github.com/galaxyproject/galaxy/pull/1688))
38+
39+
Tools may (and should) now declare a `profile` version (e.g.
40+
`<tool profile="16.04" ...>`).
41+
42+
This allows Galaxy to fire a warning if a tool uses features too new for the
43+
current version and allows us to migrate away from some undesirable default
44+
behaviors that were required for backward compatiblity.
45+
46+
---
47+
48+
#### `set -e` by default ([d020522](https://github.com/galaxyproject/galaxy/pull/1688/commits/d020522650a9bfc86c22923a01fd5d7c07c65326))
49+
50+
From the [IUC best practices documentation](http://galaxy-iuc-standards.readthedocs.org/en/latest/best_practices/tool_xml.html#command-tag):
51+
52+
> _"If you need to execute more than one shell command, concatenate them with a double ampersand (`&&`), so that an error in a command will abort the execution of the following ones."_
53+
54+
The job script generated with profile `16.04`+ tools will include a `#set -e` statement causing this behavior by default.
55+
56+
Older-style tools can enable this behavior by setting `strict="true"` on
57+
the tool `command` XML block.
58+
59+
---
60+
61+
#### Using Exit Codes for Error Detection ([b92074e](b92074e6ff87a19133b4d973577779c4ee6286d7))
62+
63+
Previously the default behavior was for Galaxy to ignore exit codes and
64+
declare a tool in error if issued any output on standard error. This was
65+
a regretable default behavior and so all tools were encouraged to declare
66+
special XML blocks to force the use of exit codes.
67+
68+
For any tool that declares a profile version of `16.04` or greater, the
69+
default is now just to use exit codes for error detection.
70+
71+
---
72+
73+
#### Unrobust Features Removed ([b92074e](b92074e6ff87a19133b4d973577779c4ee6286d7))
74+
75+
A few tool features have ben removed from tools that declare a version of `16.04` or newer.
76+
77+
- The `interepreter=` attribute on `command` blocks has been eliminated. Please use `$__tool_directory__` from within the tool instead.
78+
- `format="input"` on output datasets has been eliminated, please use `format_source=` to specify an exact input to derive datatype from.
79+
- Disables extra output file discovery by default, tools must explicitly describe the outputs to collect with `discover_dataset` tags.
80+
- Tools require a `version` attribute - previously an implicit default to `1.0.0` would be used.
81+
- `$param_file` has been eliminated.
82+
83+
---
84+
85+
#### Clean Working Directories
86+
87+
Previously, Galaxy would fill tool working directories with files related to
88+
metadata and job metric collection. Tools will no longer be executed in the same directory as these files.
89+
90+
This applies to all tools not just profile `16.04`+ tools.
91+
92+
---
93+
94+
### 16.01
95+
96+
Full [Galaxy changelog](https://docs.galaxyproject.org/en/master/releases/16.01_announce.html).
97+
98+
---
99+
100+
#### Conda Dependency Resolution ([PR #1345](https://github.com/galaxyproject/galaxy/pull/1345))
101+
102+
```xml
103+
<tool>
104+
...
105+
<requirements>
106+
<requirement type="package" version="0.11.4">FastQC</requirement>
107+
</requirements>
108+
...
109+
</tool>
110+
```
111+
112+
- Dependency resolvers tell Galaxy how to translate requirements into jobs.
113+
- The Conda dependency resolver forces Galaxy to create a conda environment
114+
for the job with `FastQC` at version `0.11.4` installed.
115+
- Only dependency resolver that can be installed at runtime - great for
116+
Docker images, heterogeneous clusters, and testing tools.
117+
- Links [Conda](http://conda.pydata.org/docs/) and [BioConda](https://bioconda.github.io/)
118+
119+
---
120+
121+
#### ToolBox Enhancements - Labels ([PR #1012](https://github.com/galaxyproject/galaxy/pull/1012))
122+
123+
![ToolBox Labels](images/gx_toolbox_labels.png)
124+
125+
---
126+
127+
#### ToolBox Enhancements - Monitoring ([PR #1398](https://github.com/galaxyproject/galaxy/pull/1398))
128+
129+
- The Galaxy toolbox can be reloaded from the Admin interface.
130+
- Tool conf files (e.g. `tool_conf.xml`) can be monitored and automatically
131+
reloaded by Galaxy.
132+
- Tool conf files can now be specified as YAML (or JSON).
133+
134+
---
135+
136+
#### Process Inputs as JSON ([PR #1405](https://github.com/galaxyproject/galaxy/pull/1405))
137+
138+
```xml
139+
<command>python "$__tool_directory/script.py" "$json_inputs"</command>
140+
<configfiles>
141+
<inputs name="json_inputs" />
142+
</configfiles>
143+
```
144+
145+
This will produce a file referenced as `$json_inputs` that contains a nested
146+
JSON structure corresponding to the tools inputs. Of limitted utility for
147+
simple command-line tools - but complex tools with many repeats, conditional,
148+
and nesting could potentially benefit from this.
149+
150+
For instance, the [JBrowse](https://github.com/galaxyproject/tools-iuc/blob/master/tools/jbrowse/jbrowse.xml)
151+
tool generates a complex JSON data structure using a `configfile` inside the
152+
XML. This is a much more portable way to deal with that.
153+
154+
---
155+
156+
157+
#### Collections
158+
159+
- `data_collection` tool parameters (`param`s) can now specify multiple
160+
`collection_type`s for consumption ([PR #1308](https://github.com/galaxyproject/galaxy/pull/1308)).
161+
- This mirrors the `format` attribute which allows a comma-separated list
162+
of potential format types.
163+
- Multiple collections can now be supplied to a `multiple="true"` data parameter ([PR #805](https://github.com/galaxyproject/galaxy/pull/805)).
164+
- Output collections can specify a `type_source` attribute (again mirroring
165+
`format_source`) ([PR #1153](https://github.com/galaxyproject/galaxy/pull/1153)).
166+
167+
---
168+
169+
### 15.10
170+
171+
Full [Galaxy changelog](https://docs.galaxyproject.org/en/master/releases/15.10_announce.html).
172+
173+
---
174+
175+
#### Collections
176+
177+
- Tools may now produce explicit nested outputs [PR #538](https://github.com/galaxyproject/galaxy/pull/538).
178+
This enhances the `discover_dataset` XML tag to allow this.
179+
- Allow certain `output` actions on collections.
180+
[PR #544](https://github.com/galaxyproject/galaxy/pull/544).
181+
- Allow `discover_dataset` tags to use `format` instead of `ext`
182+
when referring to datatype extensions/formats.
183+
- Allow `min`/`max` attributes on multiple data input parameters [PR #765](https://github.com/galaxyproject/galaxy/pull/765).
184+
185+
---
186+
187+
#### Whitelist Tools that Generate HTML ([PR #510](https://github.com/galaxyproject/galaxy/pull/510))
188+
189+
Galaxy now contains a plain text file that contains a list of tools whose
190+
output can be trusted when rendering HTML.
191+
192+
---
193+
194+
### 15.07
195+
196+
Full [Galaxy changelog](https://docs.galaxyproject.org/en/master/releases/15.07_announce.html).
197+
198+
---
199+
200+
#### Parameterized XML Macros ([PR #362](https://github.com/galaxyproject/galaxy/pull/362))
201+
202+
Macros now allow defining tokens to be consumed
203+
as XML attributes. For instance, the following definition
204+
205+
```xml
206+
<tool>
207+
<expand macro="inputs" foo="hello" />
208+
<expand macro="inputs" foo="world" />
209+
<expand macro="inputs" />
210+
<macros>
211+
<xml name="inputs" token_foo="the_default">
212+
<inputs>@FOO@</inputs>
213+
</xml>
214+
</macros>
215+
</tool>
216+
```
217+
218+
would expand out as
219+
220+
```xml
221+
<tool>
222+
<inputs>hello</inputs>
223+
<inputs>world</inputs>
224+
<inputs>the_default</inputs>
225+
</tool>
226+
```
227+
228+
---
229+
230+
#### Tool Form
231+
232+
The workflow editor was updated to the use Galaxy's newer
233+
frontend tool form.
234+
235+
![New Workflow Editor](images/gx_new_workflow_editor.png)
236+
237+
---
238+
239+
#### Environment Variables ([PR #395](https://github.com/galaxyproject/galaxy/pull/395))
240+
241+
Tools may now use `inputs` to define environment variables that will be
242+
set during tool execution. The new `environment_variables` XML block is
243+
used to define this.
244+
245+
```xml
246+
<tool>
247+
...
248+
<command>
249+
echo "\$INTVAR" > $out_file1;
250+
echo "\$FORTEST" >> $out_file1;
251+
</command>
252+
<environment_variables>
253+
<environment_variable name="INTVAR">$inttest</environment_variable>
254+
<environment_variable name="FORTEST">#for i in ['m', 'o', 'o']#$i#end for#</environment_variable>
255+
</environment_variables>
256+
...
257+
```
258+
259+
[Test tool](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/environment_variables.xml) demonstrating the use of the `environment_variables` tag.
260+
261+
---
262+
263+
#### Collections
264+
265+
- Explicit output collections can now be used in workflows. ([PR #311](https://github.com/galaxyproject/galaxy/pull/311))
266+
- The `filter` tag has been implemented for output dataset collections ([PR #455](https://github.com/galaxyproject/galaxy/pull/455). See the example tool [output_collection_filter.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/output_collection_filter.xml).
267+
268+
269+
</textarea>
270+
<script src="https://gnab.github.io/remark/downloads/remark-latest.min.js">
271+
</script>
272+
<script>
273+
var slideshow = remark.create();
274+
</script>
275+
</body>
276+
</html>

0 commit comments

Comments
 (0)