Errors in the Akri-on-Krustlet demo #4
Comments
|
@rupipal great to hear you are trying it out. Based on where the panic occurred, it looks like it is having difficulties finding the Akri configuration CRD. Just to double check, did you install Akri and the Controller in the previous step? You can confirm that the configuration CRD has been applied to the cluster via helm with |
|
Hi @kate-goldenring , Now this was my kind cluster (cluster-1) to begin with. $ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE Upon starting the Krustlet node, this is what I got. $ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE The Krustlet node was deployed. $ kubectl get no The Arkr controller gets deployed too. $ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE The gRPC proxy successfully connects with the Akri Agent and the input file seems to be written. [2021-12-18T16:19:12Z INFO dh_grpc_proxy] gRPC proxy running named as: debugEcho! However, besides those two kindnet pods not coming up, the broker Wasm Pod doesn't come up either. $ kubectl get pods -A Did spent a lot of time to look out for any missing steps. Here is where I am now :) Rupinder |
|
Hi @rupipal, that's definitely a lot of progress. Did you deploy the debug echo discovery handler yaml from this step. Your flow above is very descriptive and that step isnt in there, so I just wanted to check. I would at least expect erroring pods, since all that step is doing is deploying a standard Kubernetes Pod. |
|
Yes, defintely. That's what causes (wasi-debug-echo 0/1 Registered 0 25m ) to show up. So I'm trying to figure out what would be the id for But even at 25m, it doesn't run. |
|
@rupipal do the logs of the agent show any issue creating the device plugins? Maybe an issue around creating a socket? The Agent may need to be run privileged |
|
Here are the logs of the agent. They don't seem to show any issue. I tried with sudo; if I recall correctly, it starts looking for Kubeconfig in the root and that it can't find there.
Here are the logs of the gRPC proxy.
|
|
looks like the proxy and agent are running correctly. The Wasm debug echo discovery handler is not correctly reading the input file and writing to the output file. Can you share the logs of the debug echo discovery handler that was deployed in this step? |
|
Hi @kate-goldenring kubectl get akrii,pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES |
|
Commenting here to revive this investigation. I will be unavailable for the next couple weeks but i will see if i can find a slot of time to rerun the demo and possibly repro the issue. @rodz in case you have time to debug |
|
Thanks, Kate.
…On Tue, May 10, 2022, 8:07 PM Kate Goldenring ***@***.***> wrote:
Commenting here to revive this investigation. I will be unavailable for
the next couple weeks but i will see if i can find a slot of time to rerun
the demo and possibly repro the issue. @rodz <https://github.com/rodz> in
case you have time to debug
—
Reply to this email directly, view it on GitHub
<#4 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AGJIEWFJQHKKOQWBYWD6NADVJJYBNANCNFSM5KI3ZROQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Hi,
Though I'd have liked to check the demo ( https://github.com/project-akri/akri-on-krustlet/blob/main/demo-krustlet.md ) on k3d ( ealier I could install Akri on k3d without any major issues ( project-akri/akri#438 ), I faced errors in installing Kruslet node itself. Maybe that needs to be taken up with Krustlet people. So I switched to kind. I can see the kruslet-wasi node in the cluster. However, I seem to have hit some error.
/akri$ RUST_LOG=info RUST_BACKTRACE=1 KUBECONFIG=/.kube/config DISCOVERY_HANDLERS_DIRECTORY=~/akri AGENT_NODE_NAME=krustlet-wasi HOST_CRICTL_PATH=/usr/local/bin/crictl HOST_RUNTIME_ENDPOINT=/usr/local/bin/containerd HOST_IMAGE_ENDPOINT=/usr/local/bin/containerd target/release/agentakri.sh Agent start
akri.sh KUBERNETES_PORT found ... env_logger::init
[2021-12-17T13:05:31Z INFO akri_shared::akri::metrics] starting metrics server on port 8080 at /metrics
[2021-12-17T13:05:31Z INFO agent::util::registration] internal_run_registration_server - entered
[2021-12-17T13:05:31Z INFO agent::util::config_action] do_config_watch - enter
[2021-12-17T13:05:31Z INFO warp::server] Server::run; addr=0.0.0.0:8080
[2021-12-17T13:05:31Z INFO warp::server] listening on http://0.0.0.0:8080
[2021-12-17T13:05:31Z WARN kube::client] Unsuccessful data error parse: 404 page not found
thread 'tokio-runtime-worker' panicked at 'called
Result::unwrap()on anErrvalue: "404 page not found\n": Failed to parse error data', agent/src/main.rs:88:14stack backtrace:
0: rust_begin_unwind
at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/panicking.rs:517:5
1: core::panicking::panic_fmt
at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/core/src/panicking.rs:101:14
2: core::result::unwrap_failed
at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/core/src/result.rs:1617:5
3: <core::future::from_generator::GenFuture as core::future::future::Future>::poll
4: tokio::runtime::task::harness::Harness<T,S>::poll
5: std::thread::local::LocalKey::with
6: tokio::runtime::thread_pool::worker::Context::run_task
7: tokio::runtime::thread_pool::worker::Context::run
8: tokio::macros::scoped_tls::ScopedKey::set
9: tokio::runtime::thread_pool::worker::run
10: tokio::loom::std::unsafe_cell::UnsafeCell::with_mut
11: tokio::runtime::task::harness::Harness<T,S>::poll
12: tokio::runtime::blocking::pool::Inner::run
note: Some details are omitted, run with
RUST_BACKTRACE=fullfor a verbose backtrace.Error: JoinError::Panic(...)
The step previous to this showed:
~/akri$ cargo build -p agent --release
Updating git repository
https://github.com/kate-goldenring/h2Updating git repository
https://github.com/DazWilkin/openapi-admission-v1Downloaded crypto-mac v0.8.0
Downloaded darling v0.12.4
Downloaded float-cmp v0.8.0
...
...
...
Compiling kube-runtime v0.59.0
Compiling akri-shared v0.7.11 (/home/rupinder/akri/shared)
warning: irrefutable
while letpattern--> discovery-utils/src/discovery/mod.rs:231:27
|
231 | while let item = uds.accept().map_ok(|(st, _)| unix_stream::UnixStream(st)).await {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note:
#[warn(irrefutable_let_patterns)]on by default= note: this pattern will always match, so the loop will never exit
= help: consider instead using a
loop { ... }with aletinside itCompiling akri-debug-echo v0.7.11 (/home/rupinder/akri/discovery-handlers/debug-echo)
warning:
akri-discovery-utils(lib) generated 1 warningwarning: irrefutable
while letpattern--> agent/src/util/registration.rs:189:19
|
189 | while let item = uds.accept().map_ok(|(st, _)| unix_stream::UnixStream(st)).await {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note:
#[warn(irrefutable_let_patterns)]on by default= note: this pattern will always match, so the loop will never exit
= help: consider instead using a
loop { ... }with aletinside itwarning: irrefutable
while letpattern--> agent/src/util/device_plugin_builder.rs:143:27
|
143 | while let item = uds.accept().map_ok(|(st, _)| unix_stream::UnixStream(st)).await {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this pattern will always match, so the loop will never exit
= help: consider instead using a
loop { ... }with aletinside itwarning:
agent(bin "agent") generated 2 warningsFinished release [optimized] target(s) in 1m 46s
regards
Rupinder
The text was updated successfully, but these errors were encountered: