Croatia not coloured green in overview map

[stefanb]

Describe the bug

Croatia is not marked as covered on the overview map, even though it was added in openaddresses/openaddresses#7172 and is being covered in https://batch.openaddresses.io/location/862

To Reproduce
Steps to reproduce the behavior:

1. Go to https://batch.openaddresses.io/data
2. Scroll down to Individual Sources map
3. See Croatia in white instead of green

Expected behavior
Croatia should be coloured green on the map

Screenshots
https://batch.openaddresses.io/data

https://batch.openaddresses.io/location/862

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Additional context
There was a similar problem with Kosovo openaddresses/machine#700 but that was a bit more politically problematic area.

[iandees]

I believe the tiles that make up the map with highlighted regions is heavily cached. It might be that those need to get flushed?

[stefanb]

Cache seems to be very heavy indeed... 😄

[iandees]

Yes – I think it's an indefinite cache and I'm not sure how to clear it. @ingalls any ideas here?

[mnalis]

I've tried to debug from the client side... By doing partial blocking in firefox developer console, it seems that the stale data is coming from https://batch.openaddresses.io/api/map/0/0/0.mvt (and other closer zooms too).

Unfortunately the client-facing web server does not provide Last-Modified: header, so we don't know when it was updated, but it does say it is coming from Cloudflare CDN, which however claims (via CF-Cache-Status: DYNAMIC) that it did not consider data eligible for caching and has fetched it from origin web server.

Which in turn would seem to indicate that perhaps those global .mvt are not being updated?

HTTP header details

% wget -S https://batch.openaddresses.io/api/map/0/0/0.mvt
--2024-12-08 15:22:06--  https://batch.openaddresses.io/api/map/0/0/0.mvt
Resolving batch.openaddresses.io (batch.openaddresses.io)... 188.114.97.3, 188.114.96.3, 2a06:98c1:3121::3, ...
Connecting to batch.openaddresses.io (batch.openaddresses.io)|188.114.97.3|:443... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Sun, 08 Dec 2024 14:22:06 GMT
  Content-Type: application/vnd.mapbox-vector-tile
  Transfer-Encoding: chunked
  Connection: keep-alive
  Vary: Origin
  Access-Control-Allow-Credentials: true
  cache-control: no-transform
  CF-Cache-Status: DYNAMIC
  Report-To: {"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=Gis1VC1ZCTHjjB7q1WVDj9b%2Bk2NY6AkOKMiU0N26wC%2Fi4Fg0RAxavlUEE0%2BKHLjLBoxP1B445Tr1ZvDL0DI27U4yrvONniJQUU5Yuea8WK03fY%2FxFHyR8iWzVBpNHPyPgcrQCugw%2FBWA"}],"group":"cf-nel","max_age":604800}
  NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
  Server: cloudflare
  CF-RAY: 8eed67dbf8d95b33-VIE
  alt-svc: h3=":443"; ma=86400
  server-timing: cfL4;desc="?proto=TCP&rtt=15797&min_rtt=14760&rtt_var=6276&sent=7&recv=7&lost=0&retrans=0&sent_bytes=3126&recv_bytes=773&delivery_rate=294308&cwnd=252&unsent_bytes=0&cid=d538afcf1fbddb5e&ts=325&x=0"
Length: unspecified [application/vnd.mapbox-vector-tile]

The fact it colors normally when specifying county directly (e.g. via https://batch.openaddresses.io/location/862) seems not related to the issue, as that seems to use MultiPolygon GeoJSON to color green the specified country (e.g. via https://batch.openaddresses.io/api/map/862) regardless if it has any data in openaddresses or not.

---

TL;DR: someone with access to the server would probably need to check why those .mvt files are seemingly not being regenerated automatically when new country sources are added?

[mnalis]

Croatia is still not visible on the map almost a year later, so I'd guess it is not any caching, but something is definitely broken...

[iandees]

The query that generates these tiles returns data saying there is addresses for Croatia, but the vector tiles via the endpoint (e.g. https://batch.openaddresses.io/api/map/5/17/11.mvt) don't set the boolean flag indicating there are addresses for Croatia.

This tells me some cache has this data stored somewhere. The only cache I see in our code is here:

batch/api/routes/map.js

Lines 47 to 49 in 58a3663

	const tile = await config.cacher.get(Cacher.Miss(req.query, `tile-border-${req.params.z}-${req.params.x}-${req.params.y}`), async () => {
	return await config.borders.tile(req.params.z, req.params.x, req.params.y);
	}, false);

This is the Memcache client that connects to a memcache running on the same instance as the web server. It should get destroyed when the instance gets destroyed, which happened over the weekend.

There is also a Cloudflare cache in front of the web app. I purged the tile mentioned above and will give it a few minutes to see if anything changes.

Beyond this, I don't know what else to do. @ingalls if you have any ideas please raise them.