-
Notifications
You must be signed in to change notification settings - Fork 372
Add docu for deployers and a simple example #2946
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
5f8e71e
8220988
43c14b1
d71b4bb
5f5c892
721372a
c6f43d3
72ed52a
0aaecd0
1b1363f
9d501ef
12fac7a
80ed7f1
26da31a
f3a31e1
21f66b4
faf1365
27448c7
f774c5e
81c0993
125bba4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
.. _examples_extensions_deployers: | ||
|
||
|
||
|
||
|
||
Custom deployers | ||
================ | ||
|
||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
|
||
|
||
sources/custom_deployer_sources |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
.. examples_extensions_deployers_sources: | ||
|
||
Copy sources from all your dependencies | ||
======================================= | ||
|
||
|
||
|
||
Please, first of all, clone the sources to recreate this project. You can find them in the | ||
`examples2.0 repository <https://github.com/conan-io/examples2>`_ in GitHub: | ||
|
||
.. code-block:: bash | ||
|
||
$ git clone https://github.com/conan-io/examples2.git | ||
$ cd examples2/examples/extensions/deployers/sources | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I get:
is it expected? or typo in path? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, this is expected as the PR in examples2 has not yet been merged |
||
|
||
|
||
In this example we are going to see how to create and use a custom deployer. | ||
This deployer copies all the source files from your dependencies and puts them in a specific output folder | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
.. note:: | ||
|
||
To better understand this example, it is highly recommended to have previously read the :ref:`Deployers <reference_extensions_deployer_direct_deploy>` reference. | ||
|
||
|
||
Locate the deployer | ||
------------------- | ||
|
||
In this case, the deployer is located in the same directory than our example conanfile, | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
but as show in :ref:`Deployers <reference_extensions_deployer_direct_deploy>` reference, | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Conan will look for the specified deployer in a few extra places in order, namely: | ||
|
||
#. Absolute paths | ||
#. Relative to cwd | ||
#. In the ``[CONAN_HOME]/extensions/deploy`` folder | ||
#. As built-in deployers | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
Run it | ||
------ | ||
|
||
For our example, we have a simple recipe that only lists ``zlib`` as a requirement. | ||
With the help of the ``tools.build:download_source=True`` conf, we can force the invocation of its ``source()`` method, | ||
which will ensure that sources are available even if no build needs to be carried out. | ||
|
||
Now, you should be able to use the new deployer in both ``conan install`` and ``conan graph`` commands for any given recipe: | ||
|
||
.. code-block:: bash | ||
|
||
$ conan graph info . -c tools.build:download_source=True --deploy=sources_deploy | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
|
||
Inspecting the command output we can see that it copied the sources of all our dependencies (direct and transitive) | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
to a ``dependency_sources`` folder. After this, extra preprocessing could be accomplished to more specific needs. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. where is this |
||
|
||
Code tour | ||
--------- | ||
|
||
The **source_deploy.py** file has the following code: | ||
|
||
|
||
**sources_deploy.py** | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
.. code-block:: python | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
from conan.tools.files import copy | ||
import os | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think, as it's for conan 2 and python 3+, it may already use |
||
|
||
|
||
def deploy(graph, output_folder): | ||
for name, dep in graph.root.conanfile.dependencies.items(): | ||
copy(graph.root.conanfile, "*", dep.folders.source_folder, os.path.join(output_folder, "dependency_sources", str(dep))) | ||
|
||
|
||
deploy() | ||
++++++++ | ||
|
||
The ``deploy()`` method is called by Conan, and gets both a dependency graph and an output folder path as arguments. | ||
It iterates all the dependencies of our recipe, and copies every source file to their respective folders | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
under ``dependency_sources`` using :ref:`conan.tools.copy<conan_tools_files_copy>`. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,76 @@ | ||
.. _reference_extensions_deployers: | ||
|
||
Deployers | ||
--------- | ||
========= | ||
|
||
Deployers are a mechanism to facilitate copying files form one folder, usually the Conan cache, to user folders. | ||
While Conan provides two built-in ones (``full_deploy`` and ``direct_deploy``), users can easily manage their own | ||
with ``conan config install``. | ||
|
||
Deployers run before generators, and they can change the target folders. | ||
For example, if the ``--deploy=full_deploy`` deployer runs before ``CMakeDeps``, | ||
the files generated by ``CMakeDeps`` will point to the local copy in the user folder done by the ``full_deploy`` deployer, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why do they need to affect on generators behavior? that's super confusing part. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That is one of most requested behaviors for deployers and generators. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. that's kinda surprising behavior that deployers may modify files generated by generators. can it be made more explicit? e.g. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is kinda unspecified, what happens in case of multiple deployers, if all of them want to change target directories, what happens? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In this case, the last one would win |
||
and not to the Conan cache. Multiple deployers can be specified by supplying more than one ``--deploy=`` argument, | ||
and they will be ran in order of appearance. | ||
|
||
Deployers can be multi-configuration. Running ``conan install . --deploy=full_deploy`` repeatedly for different profiles | ||
can achieve a fully self-contained project, including all the artifacts, binaries, and build files. | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This project would be completely independent of Conan and no longer require it at all to build. | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
Built-in deployers | ||
------------------ | ||
|
||
.. _reference_extensions_deployer_full_deploy: | ||
|
||
full_deploy() | ||
^^^^^^^^^^^^^ | ||
|
||
Deploys each package folder of every dependency to your recipe's ``output_folder`` in a subfolder tree based on: | ||
|
||
#. The build context | ||
#. The dependency name and version | ||
#. The build type | ||
#. The build arch | ||
|
||
Such that every dependency will end up in a folder such as: | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
``[OUTPUT_FOLDER]/host/dep/0.1/Release/x86_64`` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. as it doesn't include all the settings/options, is there a risk that several deployments will overwrite each other? maybe it's better just to use package id here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is, but if the user is worried about such problems, they are encouraged to write their own deployer, but let's tag @memsharded in case he thinks this could be nice to have |
||
|
||
|
||
.. _reference_extensions_deployer_direct_deploy: | ||
|
||
direct_deploy() | ||
^^^^^^^^^^^^^^^ | ||
|
||
Same as ``full_deploy()```, but only processes your recipe's *direct* dependencies. | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
Custom deployers | ||
---------------- | ||
|
||
Custom deployers can be managed via ``conan config install``. When looking for a specific deployer, | ||
Conan will look in these locations for the deployer in the following order: | ||
|
||
#. Absolute paths | ||
#. Relative to cwd | ||
#. In the ``[CONAN_HOME]/extensions/deploy`` folder | ||
#. As built-in deployers | ||
|
||
For each installed file, Conan will look for a ``deploy()`` method which to call. | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
The function signature of your custom deployers should be as follows: | ||
|
||
**my_custom_deployer.py** | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
.. code-block:: python | ||
AbrilRBS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
def deploy(graph, output_folder: str): | ||
|
||
(Note that the arguments are passed as named parameters, so both the ``graph`` and ``output_folder`` names are mandatory) | ||
franramirez688 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
You can access your conanfile object with ``graph.root.conanfile``. | ||
See :ref:`ConanFile.dependencies<conan_conanfile_model_dependencies>` for information on how to iterate over its dependencies. | ||
Your custom deployer can now be invoked as if it were a built-in deployer using the filename in which it's found, | ||
in this case ``conan install . --deploy=my_custom_deployer``. Note that supplying the **.py** extension is optional. | ||
|
||
See the :ref:`custom deployers<examples_extensions_deployers>` section for examples on how to implement your own deployers. |
Uh oh!
There was an error while loading. Please reload this page.