1
1
# Multi-processing and Distributed Computing
2
2
3
- An implementation of distributed memory parallel computing is provided by module ` Distributed `
3
+ An implementation of distributed memory parallel computing is provided by module [ ` Distributed ` ] ( @ ref man-distributed)
4
4
as part of the standard library shipped with Julia.
5
5
6
6
Most modern computers possess more than one CPU, and several computers can be combined together
@@ -45,11 +45,11 @@ computation is running on the worker.
45
45
46
46
Let's try this out. Starting with ` julia -p n ` provides ` n ` worker processes on the local machine.
47
47
Generally it makes sense for ` n ` to equal the number of CPU threads (logical cores) on the machine. Note that the ` -p `
48
- argument implicitly loads module ` Distributed ` .
48
+ argument implicitly loads module [ ` Distributed ` ] ( @ ref man-distributed) .
49
49
50
50
51
51
``` julia
52
- $ ./ julia - p 2
52
+ $ julia - p 2
53
53
54
54
julia> r = remotecall (rand, 2 , 2 , 2 )
55
55
Future (2 , 1 , 4 , nothing )
@@ -198,7 +198,7 @@ loaded
198
198
```
199
199
200
200
As usual, this does not bring ` DummyModule ` into scope on any of the process, which requires
201
- ` using ` or ` import ` . Moreover, when ` DummyModule ` is brought into scope on one process, it
201
+ [ ` using ` ] ( @ref ) or [ ` import ` ] ( @ref ) . Moreover, when ` DummyModule ` is brought into scope on one process, it
202
202
is not on any other:
203
203
204
204
``` julia-repl
@@ -236,7 +236,7 @@ like a process providing an interactive prompt.
236
236
237
237
Finally, if ` DummyModule.jl ` is not a standalone file but a package, then `using
238
238
DummyModule` will _load_ ` DummyModule.jl` on all processes, but only bring it into scope on
239
- the process where ` using ` was called.
239
+ the process where [ ` using ` ] ( @ref ) was called.
240
240
241
241
## Starting and managing worker processes
242
242
@@ -262,7 +262,7 @@ julia> addprocs(2)
262
262
3
263
263
```
264
264
265
- Module ` Distributed ` must be explicitly loaded on the master process before invoking [ ` addprocs ` ] ( @ref ) .
265
+ Module [ ` Distributed ` ] ( @ ref man-distributed) must be explicitly loaded on the master process before invoking [ ` addprocs ` ] ( @ref ) .
266
266
It is automatically made available on the worker processes.
267
267
268
268
Note that workers do not run a ` ~/.julia/config/startup.jl ` startup script, nor do they synchronize
@@ -322,8 +322,8 @@ is replaced with a more expensive operation. Then it might make sense to add ano
322
322
statement just for this step.
323
323
324
324
## Global variables
325
- Expressions executed remotely via ` @spawnat ` , or closures specified for remote execution using
326
- ` remotecall ` may refer to global variables. Global bindings under module ` Main ` are treated
325
+ Expressions executed remotely via [ ` @spawnat ` ] ( @ref ) , or closures specified for remote execution using
326
+ [ ` remotecall ` ] ( @ref ) may refer to global variables. Global bindings under module ` Main ` are treated
327
327
a little differently compared to global bindings in other modules. Consider the following code
328
328
snippet:
329
329
@@ -335,7 +335,7 @@ remotecall_fetch(()->sum(A), 2)
335
335
In this case [ ` sum ` ] ( @ref ) MUST be defined in the remote process.
336
336
Note that ` A ` is a global variable defined in the local workspace. Worker 2 does not have a variable called
337
337
` A ` under ` Main ` . The act of shipping the closure ` ()->sum(A) ` to worker 2 results in ` Main.A ` being defined
338
- on 2. ` Main.A ` continues to exist on worker 2 even after the call ` remotecall_fetch ` returns. Remote calls
338
+ on 2. ` Main.A ` continues to exist on worker 2 even after the call [ ` remotecall_fetch ` ] ( @ref ) returns. Remote calls
339
339
with embedded global references (under ` Main ` module only) manage globals as follows:
340
340
341
341
- New global bindings are created on destination workers if they are referenced as part of a remote call.
@@ -1205,12 +1205,12 @@ requirements for the inbuilt `LocalManager` and `SSHManager`:
1205
1205
Securing and encrypting all worker-worker traffic (via SSH) or encrypting individual messages
1206
1206
can be done via a custom ` ClusterManager ` .
1207
1207
1208
- * If you specify ` multiplex=true ` as an option to ` addprocs ` , SSH multiplexing is used to create
1208
+ * If you specify ` multiplex=true ` as an option to [ ` addprocs ` ] ( @ref ) , SSH multiplexing is used to create
1209
1209
a tunnel between the master and workers. If you have configured SSH multiplexing on your own and
1210
1210
the connection has already been established, SSH multiplexing is used regardless of ` multiplex `
1211
1211
option. If multiplexing is enabled, forwarding is set by using the existing connection
1212
1212
(` -O forward ` option in ssh). This is beneficial if your servers require password authentication;
1213
- you can avoid authentication in Julia by logging in to the server ahead of ` addprocs ` . The control
1213
+ you can avoid authentication in Julia by logging in to the server ahead of [ ` addprocs ` ] ( @ref ) . The control
1214
1214
socket will be located at ` ~/.ssh/julia-%r@%h:%p ` during the session unless the existing multiplexing
1215
1215
connection is used. Note that bandwidth may be limited if you create multiple processes on a node
1216
1216
and enable multiplexing, because in that case processes share a single multiplexing TCP connection.
@@ -1236,7 +1236,7 @@ For example, cookies can be pre-shared and hence not specified as a startup argu
1236
1236
1237
1237
## Specifying Network Topology (Experimental)
1238
1238
1239
- The keyword argument ` topology ` passed to ` addprocs ` is used to specify how the workers must be
1239
+ The keyword argument ` topology ` passed to [ ` addprocs ` ] ( @ref ) is used to specify how the workers must be
1240
1240
connected to each other:
1241
1241
1242
1242
* ` :all_to_all ` , the default: all workers are connected to each other.
0 commit comments