[BUG] CDP Still only has OLD Data and Missing New Router Data

[taariq]

@byte-bandit This issue still persists with old data from old routers.

Current (new) GARRYTAN token factory address is factory/paloma1kpqenkph603uggcgdztq76ec35guzkn08v8qvq0ytk36sepynuysnmky9f/GARRYTAN.15

However, the API returns results for this denom:
factory/paloma1z36hqznwmendmg2ks5scmw63qp3n2v0yfcnt53c6ljcg344uw5rsqk62gv/GARRYTAN.15
api result is old data

https://cdp.palomachain.com/docs#/Advanced%20Charts/rest%2Fv1.SymbolsInteractor shows:

{
  "symbols": [
    {
      "description": "BONDING: [factory/paloma1kpf02jzwyhkzdve2pcdt6lq7zns997lung22qscql69g6eehvedsexnegx/upusd,factory/paloma1z36hqznwmendmg2ks5scmw63qp3n2v0yfcnt53c6ljcg344uw5rsqk62gv/GARRYTAN.15]",
      "exchange": "BONDING",
      "symbol": "BONDING:UPUSD-1kpf02/GARRYTAN.15-1z36hq",
      "ticker": "UPUSD-1kpf02/GARRYTAN.15-1z36hq",
      "type": "crypto"
    }
  ]
}

[byte-bandit]

The indexed data will always contain historical data, meaning when you do a search for GARRYTAN.15, it will always show the old data (among with any other, newer tokens).

It should show the new token as well though. I can see that it hasn't been detected yet by the CDP. Can you share one or more transactions with trading data for factory/paloma1kpqenkph603uggcgdztq76ec35guzkn08v8qvq0ytk36sepynuysnmky9f/GARRYTAN.15 ?

[taariq]

@byte-bandit here are 2 purchases that were made on Polygon. https://polygonscan.com/token/0xFD5fffc0d9DeC9CA7C520d59dB10276641e0544E

These purchases were made on BSC:
https://bscscan.com/token/0xca83ef94cd67cf68ace67375cfcd3679fad5dbe4

[byte-bandit]

I need the Paloma transactions, the CDP does not index any off chain data.

[taariq]

I need the Paloma transactions, the CDP does not index any off chain data.

@fullstack412 are you able to help with the Paloma Transactions for trades on GARRYTAN.15?

[fullstack412]

https://paloma.explorers.guru/transaction/4CD10E4C733F5597EC23C9E4B2F02FCAD799B0FF9AA7F37CC8BA9A3549BC55E9
https://paloma.explorers.guru/transaction/876A8A824A051460BE2407DD30114D9DD05FFF96B12E63A7CFF68B0620A85915

@byte-bandit

[byte-bandit]

@fullstack412 Neither of these contain a swap event. Only the contents of swap events are indexed. If the transaction does not contain a this event, it will not be indexed.

[fullstack412]

@wc117

[taariq]

From @byte-bandit

The CDP doesn't inspect message types contained in a TX. It looks at the event data of the final, applied transaction, and only at the event data. And if there is no event from the wasm module called swap, the CDP will not pick up the TX.

[taariq]

From @wc117 to @byte-bandit

"The transactions Jacky shared are swap transactions, and the protocol must get all data from the transaction. If he can't get events from the tx, the protocol can't get any data from exchange txes by purchasers. I am not sure why he can't get events from internal messages. It looks like it worked before,e and it does not work now. But we did not make any major changes in the Purchaser logic."

[verabehr]

also @byte-bandit @taariq
can we see any past swap transactions that were ingested and a swap event was dectected successfully? Per @wc117 point, it looks like it worked before, but I'm not sure how we can see what the tx that would be

[byte-bandit]

@verabehr Here's an example TX that was ingested successfully in the past: https://paloma.explorers.guru/transaction/A03FA3599B92BFD93AF098A7AED871E6F98896A1E14DDF7ADBA8D9191BC3146E

Note the swap event present in the events[] attached to the transaction results. That is what the CDP need, and that is what's missing from the other transactions shared in this ticket.

[taariq]

From @byte-bandit

I can guarantee that this has 100% never worked.

There is nothing broken, the chain indexer works the same it has since it was launched a few weeks ago. When I built the indexer, I had some discussions with I think @wc117 , and those set some hard technical requirements. And I built the indexer on top of those hard requirements:

• requirement 1: only transactions from registered addresses will be indexed (at the moment, those are the router addresses of the bonding curve and paloma dex contracts)

• requirement 2: only transactions which which contain swap event will be parsed.

That's what we agreed on, that's what was built, and that's what has been working until now. So if I had to take a guess, it looks like the contracts which emitted those TXs you sent over earlier do not actually emit a swap event after broadcasting the message?

[wc117]

I checked both transactions. And they are completely same from CosmWasm. The former emitted all events of internal messages but the latter didn't emit events from internal messages. As far as my checking CW update, there is no update that affects to internal messages and events. I think the changes came from protocol update in recent 3 weeks. @byte-bandit @taariq

[webelf101]

I couldn't put so long time to review the protocol update but I found @byte-bandit made some update in event management part. It might affect to current changes. e504417

[webelf101]

I am curious whether the update led to miss some events from internal messages or not.

[byte-bandit]

This looks like it might be related, but I'm not convinced to be honest. I tested that event processing upgrade before pushing it out - but more importantly, there are still wasm events on recent transactions.

Here's another thought though. Can you confirm the trades actually took place? I do not see any coin_sent or coin_received events on both linked transactions either. Can you confirm that balances actually changed after those TXs? Just looking at the data, it looks like no trade took place (missing funds, out of gas, whatever).

[byte-bandit]

Also, compare other transactions, i.e. this one: https://paloma.explorers.guru/transaction/C470E7D671FB9AD313720F898779FFF45E2CF9EC85A03DBAA4E1BA233965C8CE?height=34847293

It's not a swap transaction, but all expected events are attached to the transaction (moved coins, wasm event, etc...).

[webelf101]

But actually, the transactions that you couldn't get events, made exchanges. And according to @fullstack412 we could get coins from bonding curve and successfully sent the denom to users. So the transaction worked perfectly as we expected except for events. @byte-bandit cc @taariq

[wc117]

Also, compare other transactions, i.e. this one: https://paloma.explorers.guru/transaction/C470E7D671FB9AD313720F898779FFF45E2CF9EC85A03DBAA4E1BA233965C8CE?height=34847293

It's not a swap transaction, but all expected events are attached to the transaction (moved coins, wasm event, etc...).

Yes. But all the events came from the CW contract, not from the other CW contract that was received message. It did not send message to other CW so there are not any events from the internal messages. We are missing events when we send message to the other CosmWasm and the CosmWasm emits the events. This is my idea. @byte-bandit @taariq cc @webelf101

[byte-bandit]

This should be fairly easy to debug, right? Just compile a small example CW that doesn't really do much apart from emitting events. That will help debugging this.

[taariq]

From @byte-bandit

It's possible that the upgrade broke event recording down the line, but I still do not believe this. As far as I know, there is no such thing as a call chain of smart contracts in CW. You can make a dedicated call to a different contract, but that would end up being a separate TX. In that case, the events we're looking for would simply be on another tx.

Then, we're still seeing all expected events (including coin movements) on other contract execution transactions, but we're not seeing them on the two offending txs.
Now

Something is fishy for sure, but I remember working on this code, and it is literally a black box. If it REALLY doesn't work, I need some proper, irrefutable proof. And we can that with some demo contracts. I will also need those demo contracts to verify that any solution I can cook up will actually solve the problem

Over the past weeks, a lot of the incidents brought to me turned out to be either not caused by protocol, or simply wrong usage of the API, or some other user error. I'm not saying Paloma is fool proof, but this WASM integration is very hard to debug. It will take a long time. So, I need to be sure it is actually broken.

[wc117]

https://github.com/VolumeFi/paloma-event-test-cw
This is what I built for the test.
I deployed 2 contracts - receiver and sender.
paloma1kwcf0scnk2kjgu5w7ldpa49nmcx474chclz0jcgd2h3920me7aysh2nmnx This is sender address.
paloma1hnnj5zealwnacgf58lt96zwjdmnwgp4zj243drfukywmyx5tykushuy6dm This is receiver address.
paloma17esj9dnjhnpezfqmpwdwg2yldnc3udrdv5e76w This is my wallet address.

When I run send_coin transaction with receiver address to sender contract, the funds moves from my address to sender, and moves to receiver address, and moves to my address back.

CCA3F9BB2950D9737141A6240DD6B77564934B56B11208D9D50F80CDD5E7AADB
This is transaction hash.
I sent 1 GRAIN in the tx and it moved as my wallet -> sender -> receiver -> my wallet.
I confirmed the grain balance of my wallet before and after the tx are the same.

So we should be able to 3 coin transfer events at least.

But I couldn't see any events data about receiver address paloma1hnnj5zealwnacgf58lt96zwjdmnwgp4zj243drfukywmyx5tykushuy6dm in the tx data.

@byte-bandit @taariq

[byte-bandit]

@wc117 The demo contracts proved incredibly helpful in debugging this, thank you! There was a bug in the protocol that would explicitly swallow events, but only from downstream contract calls. That's why it slipped through testing and wasn't found until now.