Skip to content

Commit d675fef

Browse files
andrewazoresandrewazores
authored
feat(discovery): Podman platform (#1394)
* tmp * rebase fixes * working on experimental podman handler * update netty native version * fixup! working on experimental podman handler * add comment with vertx deps BOM note * add note about systemctl start podman.socket * fixup! update netty native version * fixup netty version Signed-off-by: Andrew Azores <[email protected]> * add security opt flag for mounting podman socket * cleanup UID lookup, run on worker thread, set API version to 2.3 * add Podman built-in discovery plugin Signed-off-by: Andrew Azores <[email protected]> * use request query param to filter for containers with specific label * cleanup * remove obsolete podman API testing endpoints * improve JMX URL handling, add label to smoketest targets * perform fewer COW updates Signed-off-by: Andrew Azores <[email protected]> * fix ignored podless containers * rebase cleanup * refactor platform selection allow multiple available strategies to be selected. the highest priority available platform provides the authmanager if none explicitly specified. set of selected strategies can be mapped to set of available platform clients, allowing multiple enabled built-in plugins at once Signed-off-by: Andrew Azores <[email protected]> * deregister builtin plugins if they are no longer available/selected Signed-off-by: Andrew Azores <[email protected]> * CRYOSTAT_PLATFORM takes comma-separated list Signed-off-by: Andrew Azores <[email protected]> --------- Signed-off-by: Andrew Azores <[email protected]>
1 parent ef6b6bd commit d675fef

13 files changed

+576
-81
lines changed

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ Cryostat can be configured via the following environment variables:
128128
* `CRYOSTAT_CORS_ORIGIN`: the origin for CORS to load a different cryostat-web instance. Defaults to the empty string, which disables CORS.
129129
* `CRYOSTAT_MAX_WS_CONNECTIONS`: the maximum number of websocket client connections allowed (minimum 1, maximum `Integer.MAX_VALUE`, default `Integer.MAX_VALUE`)
130130
* `CRYOSTAT_AUTH_MANAGER`: the authentication/authorization manager used for validating user accesses. See the `USER AUTHENTICATION / AUTHORIZATION` section for more details. Set to the fully-qualified class name of the auth manager implementation to use, ex. `io.cryostat.net.BasicAuthManager`. Defaults to an AuthManager corresponding to the selected deployment platform, whether explicit or automatic (see below).
131-
* `CRYOSTAT_PLATFORM`: the platform client used for performing platform-specific actions, such as listing available target JVMs. If `CRYOSTAT_AUTH_MANAGER` is not specified then a default auth manager will also be selected corresponding to the platform, whether that platform is specified by the user or automatically detected. Set to the fully-qualified name of the platform detection strategy implementation to use, ex. `io.cryostat.platform.internal.KubeEnvPlatformStrategy`.
131+
* `CRYOSTAT_PLATFORM`: the platform clients used for performing platform-specific actions, such as listing available target JVMs. If `CRYOSTAT_AUTH_MANAGER` is not specified then a default auth manager will also be selected corresponding to the highest priority platform, whether those platforms are specified by the user or automatically detected. Set to the fully-qualified names of the platform detection strategy implementations to use, ex. `io.cryostat.platform.internal.KubeEnvPlatformStrategy,io.cryostat.platform.internal.PodmanPlatformStrategy`.
132132
* `CRYOSTAT_ENABLE_JDP_BROADCAST`: enable the Cryostat JVM to broadcast itself via JDP (Java Discovery Protocol). Defaults to `true`.
133133
* `CRYOSTAT_JDP_ADDRESS`: the JDP multicast address to send discovery packets. Defaults to `224.0.23.178`.
134134
* `CRYOSTAT_JDP_PORT`: the JDP multicast port to send discovery packets. Defaults to `7095`.

‎pom.xml

+8
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,14 @@
207207
<artifactId>jasypt-hibernate5</artifactId>
208208
<version>${org.jasypt-hibernate5.version}</version>
209209
</dependency>
210+
<dependency>
211+
<groupId>io.netty</groupId>
212+
<artifactId>netty-transport-native-epoll</artifactId>
213+
<classifier>linux-x86_64</classifier>
214+
<version>4.1.86.Final</version><!-- FIXME this should be tied to the vertx version -->
215+
<!-- https://repo1.maven.org/maven2/io/vertx/vertx-dependencies/${io.vertx.web.version}/vertx-dependencies-${io.vertx.web.version}.pom -->
216+
<!-- https://groups.google.com/g/vertx/c/uzzcgw-YuOg -->
217+
</dependency>
210218
<dependency>
211219
<groupId>org.slf4j</groupId>
212220
<artifactId>slf4j-jdk14</artifactId>

‎run.sh

+11
Original file line numberDiff line numberDiff line change
@@ -107,13 +107,22 @@ if ! podman pod exists cryostat-pod; then
107107
--publish "$CRYOSTAT_EXT_WEB_PORT":"$CRYOSTAT_WEB_PORT"
108108
fi
109109

110+
# do: $ podman system service -t 0
111+
# or do: $ systemctl --user start podman.socket
112+
# to create the podman.sock to volume-mount into the container
113+
#
114+
# to check the podman socket is reachable and connectable within the container:
115+
# $ podman exec -it cryo /bin/sh
116+
# sh-4.4# curl -v -s --unix-socket /run/user/0/podman/podman.sock http://d:80/v3.0.0/libpod/info
117+
#
110118
# run as root (uid 0) within the container - with rootless podman this means
111119
# that the process will actually run with your own uid on the host machine,
112120
# rather than the uid being remapped to something else
113121
podman run \
114122
--pod cryostat-pod \
115123
--name cryostat \
116124
--user 0 \
125+
--label io.cryostat.connectUrl="service:jmx:rmi:///jndi/rmi://localhost:0/jmxrmi" \
117126
--memory 768M \
118127
--mount type=bind,source="$(dirname "$0")/archive",destination=/opt/cryostat.d/recordings.d,relabel=shared \
119128
--mount type=bind,source="$(dirname "$0")/certs",destination=/certs,relabel=shared \
@@ -122,6 +131,8 @@ podman run \
122131
--mount type=bind,source="$(dirname "$0")/templates",destination=/opt/cryostat.d/templates.d,relabel=shared \
123132
--mount type=bind,source="$(dirname "$0")/truststore",destination=/truststore,relabel=shared \
124133
--mount type=bind,source="$(dirname "$0")/probes",destination=/opt/cryostat.d/conf.d/probes.d,relabel=shared \
134+
-v "$XDG_RUNTIME_DIR"/podman/podman.sock:/run/user/0/podman/podman.sock:Z \
135+
--security-opt label=disable \
125136
-e CRYOSTAT_ENABLE_JDP_BROADCAST="true" \
126137
-e CRYOSTAT_REPORT_GENERATOR="$CRYOSTAT_REPORT_GENERATOR" \
127138
-e CRYOSTAT_PLATFORM="$CRYOSTAT_PLATFORM" \

‎smoketest.sh

+4
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@ runDemoApps() {
9797
--env HTTP_PORT=8081 \
9898
--env JMX_PORT=9093 \
9999
--pod cryostat-pod \
100+
--label io.cryostat.connectUrl="service:jmx:rmi:///jndi/rmi://localhost:9093/jmxrmi" \
100101
--rm -d quay.io/andrewazores/vertx-fib-demo:0.9.1
101102

102103
podman run \
@@ -105,6 +106,7 @@ runDemoApps() {
105106
--env JMX_PORT=9094 \
106107
--env USE_AUTH=true \
107108
--pod cryostat-pod \
109+
--label io.cryostat.connectUrl="service:jmx:rmi:///jndi/rmi://localhost:9094/jmxrmi" \
108110
--rm -d quay.io/andrewazores/vertx-fib-demo:0.9.1
109111

110112
podman run \
@@ -114,6 +116,7 @@ runDemoApps() {
114116
--env USE_SSL=true \
115117
--env USE_AUTH=true \
116118
--pod cryostat-pod \
119+
--label io.cryostat.connectUrl="service:jmx:rmi:///jndi/rmi://localhost:9095/jmxrmi" \
117120
--rm -d quay.io/andrewazores/vertx-fib-demo:0.9.1
118121

119122
local webPort;
@@ -234,6 +237,7 @@ runReportGenerator() {
234237
--name reports \
235238
--pull always \
236239
--pod cryostat-pod \
240+
--label io.cryostat.connectUrl="service:jmx:remote+http://localhost:${RJMX_PORT}" \
237241
--cpus 1 \
238242
--memory 512M \
239243
--restart on-failure \

‎src/main/java/io/cryostat/discovery/BuiltInDiscovery.java

+39-19
Original file line numberDiff line numberDiff line change
@@ -37,19 +37,22 @@
3737
*/
3838
package io.cryostat.discovery;
3939

40+
import java.util.HashSet;
41+
import java.util.Iterator;
4042
import java.util.Map;
4143
import java.util.Set;
44+
import java.util.SortedSet;
4245
import java.util.UUID;
4346
import java.util.function.Consumer;
47+
import java.util.stream.Stream;
4448

45-
import io.cryostat.configuration.Variables;
4649
import io.cryostat.core.log.Logger;
47-
import io.cryostat.core.sys.Environment;
4850
import io.cryostat.messaging.notifications.NotificationFactory;
4951
import io.cryostat.platform.PlatformClient;
5052
import io.cryostat.platform.TargetDiscoveryEvent;
5153
import io.cryostat.platform.discovery.EnvironmentNode;
5254
import io.cryostat.platform.internal.CustomTargetPlatformClient;
55+
import io.cryostat.platform.internal.PlatformDetectionStrategy;
5356

5457
import dagger.Lazy;
5558
import io.vertx.core.AbstractVerticle;
@@ -60,33 +63,48 @@ public class BuiltInDiscovery extends AbstractVerticle implements Consumer<Targe
6063
static final String NOTIFICATION_CATEGORY = "TargetJvmDiscovery";
6164

6265
private final DiscoveryStorage storage;
63-
private final Set<PlatformClient> platformClients;
66+
private final Set<PlatformDetectionStrategy<?>> selectedStrategies;
67+
private final Set<PlatformDetectionStrategy<?>> unselectedStrategies;
6468
private final Lazy<CustomTargetPlatformClient> customTargets;
65-
private final Environment env;
69+
private final Set<PlatformClient> enabledClients = new HashSet<>();
6670
private final NotificationFactory notificationFactory;
6771
private final Logger logger;
6872

6973
BuiltInDiscovery(
7074
DiscoveryStorage storage,
71-
Set<PlatformClient> platformClients,
75+
SortedSet<PlatformDetectionStrategy<?>> selectedStrategies,
76+
SortedSet<PlatformDetectionStrategy<?>> unselectedStrategies,
7277
Lazy<CustomTargetPlatformClient> customTargets,
73-
Environment env,
7478
NotificationFactory notificationFactory,
7579
Logger logger) {
7680
this.storage = storage;
77-
this.platformClients = platformClients;
81+
this.selectedStrategies = selectedStrategies;
82+
this.unselectedStrategies = unselectedStrategies;
7883
this.customTargets = customTargets;
79-
this.env = env;
8084
this.notificationFactory = notificationFactory;
8185
this.logger = logger;
8286
}
8387

8488
@Override
8589
public void start(Promise<Void> start) {
8690
storage.addTargetDiscoveryListener(this);
87-
(env.hasEnv(Variables.DISABLE_BUILTIN_DISCOVERY)
88-
? Set.of(customTargets.get())
89-
: platformClients)
91+
92+
unselectedStrategies.stream()
93+
.map(PlatformDetectionStrategy::getPlatformClient)
94+
.forEach(
95+
platform ->
96+
storage.getBuiltInPluginByRealm(
97+
platform.getDiscoveryTree().getName())
98+
.map(PluginInfo::getId)
99+
.ifPresent(storage::deregister));
100+
101+
Stream.concat(
102+
// ensure custom targets is always available regardless of other
103+
// configurations
104+
Stream.of(customTargets.get()),
105+
selectedStrategies.stream()
106+
.map(PlatformDetectionStrategy::getPlatformClient))
107+
.distinct()
90108
.forEach(
91109
platform -> {
92110
logger.info(
@@ -126,6 +144,7 @@ public void start(Promise<Void> start) {
126144
try {
127145
platform.start();
128146
platform.load(promise);
147+
enabledClients.add(platform);
129148
} catch (Exception e) {
130149
start.fail(e);
131150
}
@@ -136,14 +155,15 @@ public void start(Promise<Void> start) {
136155
@Override
137156
public void stop() {
138157
storage.removeTargetDiscoveryListener(this);
139-
this.platformClients.forEach(
140-
platform -> {
141-
try {
142-
platform.stop();
143-
} catch (Exception e) {
144-
logger.error(e);
145-
}
146-
});
158+
Iterator<PlatformClient> it = enabledClients.iterator();
159+
while (it.hasNext()) {
160+
try {
161+
it.next().stop();
162+
} catch (Exception e) {
163+
logger.error(e);
164+
}
165+
it.remove();
166+
}
147167
}
148168

149169
@Override

‎src/main/java/io/cryostat/discovery/DiscoveryModule.java

+13-4
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939

4040
import java.time.Duration;
4141
import java.util.Set;
42+
import java.util.SortedSet;
4243

4344
import javax.inject.Named;
4445
import javax.inject.Singleton;
@@ -50,9 +51,10 @@
5051
import io.cryostat.core.log.Logger;
5152
import io.cryostat.core.sys.Environment;
5253
import io.cryostat.messaging.notifications.NotificationFactory;
53-
import io.cryostat.platform.PlatformClient;
54+
import io.cryostat.platform.PlatformModule;
5455
import io.cryostat.platform.discovery.AbstractNode;
5556
import io.cryostat.platform.internal.CustomTargetPlatformClient;
57+
import io.cryostat.platform.internal.PlatformDetectionStrategy;
5658
import io.cryostat.recordings.JvmIdHelper;
5759
import io.cryostat.rules.MatchExpressionEvaluator;
5860
import io.cryostat.util.PluggableTypeAdapter;
@@ -116,13 +118,20 @@ static DiscoveryStorage provideDiscoveryStorage(
116118
@Singleton
117119
static BuiltInDiscovery provideBuiltInDiscovery(
118120
DiscoveryStorage storage,
119-
Set<PlatformClient> platformClients,
121+
@Named(PlatformModule.SELECTED_PLATFORMS)
122+
SortedSet<PlatformDetectionStrategy<?>> selectedStrategies,
123+
@Named(PlatformModule.UNSELECTED_PLATFORMS)
124+
SortedSet<PlatformDetectionStrategy<?>> unselectedStrategies,
120125
Lazy<CustomTargetPlatformClient> customTargets,
121-
Environment env,
122126
NotificationFactory notificationFactory,
123127
Logger logger) {
124128
return new BuiltInDiscovery(
125-
storage, platformClients, customTargets, env, notificationFactory, logger);
129+
storage,
130+
selectedStrategies,
131+
unselectedStrategies,
132+
customTargets,
133+
notificationFactory,
134+
logger);
126135
}
127136

128137
@Provides

‎src/main/java/io/cryostat/net/HttpServer.java

+4-1
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,10 @@ public class HttpServer extends AbstractVerticle {
8282
.setPort(netConf.getInternalWebServerPort())
8383
.addWebSocketSubProtocol("*")
8484
.setCompressionSupported(true)
85-
.setLogActivity(true)));
85+
.setLogActivity(true)
86+
.setTcpFastOpen(true)
87+
.setTcpNoDelay(true)
88+
.setTcpQuickAck(true)));
8689

8790
if (!sslConf.enabled()) {
8891
this.logger.warn("No available SSL certificates. Fallback to plain HTTP.");

‎src/main/java/io/cryostat/net/NetworkModule.java

+2-1
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@
6969
import dagger.Provides;
7070
import dagger.multibindings.IntoSet;
7171
import io.vertx.core.Vertx;
72+
import io.vertx.core.VertxOptions;
7273
import io.vertx.ext.web.client.WebClient;
7374
import io.vertx.ext.web.client.WebClientOptions;
7475

@@ -155,7 +156,7 @@ static JFRConnectionToolkit provideJFRConnectionToolkit(
155156
@Provides
156157
@Singleton
157158
static Vertx provideVertx() {
158-
return Vertx.vertx();
159+
return Vertx.vertx(new VertxOptions().setPreferNativeTransport(true));
159160
}
160161

161162
@Provides

0 commit comments

Comments
 (0)