hckrnws

Next we are combining this work with an AR collection app for on-site capture and testing out workflows for "shared global anchors" (aka Spatial anchors or Cloud anchors) a recurring topic in the w3c immersive web working group: https://github.com/immersive-web/proposals/issues/82#issueco...

Really nice! I did a quick test some weeks ago and apparently using google street view images to generate Gaussian splats works quite well: https://www.instagram.com/p/C2xozxVAdpG/

(I just took screenshots of the Street View, not the API, so the ground is falling apart a bit because of the street names.)

Agreed tons of potential here, however we may need to get our own geo data for this: Google was very clear during the API quota approval process that 3d tiles cannot be used for ML training or generation of derivative assets like Gaussian splats, at least with this specific product offering.

Have you looked into Mapillary? Could be a viable alternative, even though the image quality varies a lot.

Great idea... could even be generated "on-demand" only when the user requests high fidelity for a specific area!

It's not immediately obvious, but you can move the viewpoint via panning by holding down shift while clicking and dragging (on desktop, at least). My natural instinct was to double-click like in Google Streetview.

Not sure if there's a limit on the map area but I was able to navigate all the way to Ocean Beach!

Congratulations on the amazing work/demo! An interesting extension of this would be integrating the "Level of Gaussians" approach presented at: https://zju3dv.github.io/LoG_webpage/

Also interesting would be to generate gaussian splats from Google Street View footage, and integrate it directly into the Google Maps 3D data as you have done.

Related: https://news.ycombinator.com/item?id=39909110 (April 2024)

This is the best real world (heh) demo of Guassian Splats I've seen so far.

Now to fork and enable the FPV camera controls.

Great work!

Yes, you can just remix and remove the orbit controls in lines 43 and 44 https://glitch.com/edit/#!/maps-and-splats?path=index.html%3... and then add `look-controls` `wasd-controls` to the camera entity. https://aframe.io/docs/1.5.0/components/wasd-controls.html

Thanks!

For anyone interested this is what I ended up with to fly around:

  <a-entity
    camera
    look-controls="pointerLockEnabled: true;"
    wasd-controls="enabled: true; fly: true; acceleration: 3000;"
  ></a-entity>

The video looks awesome! I'm on mobile and the first demo is loading 3D tiles but not the splats for me.

I saw the splat "stall" during loading once while testing but couldn't reproduce. Curious if you reload after a stalled splat load if it works?

This is great!

Unrelated - does anyone know a great library for building tile maps for 2D scatter plots?

Check out tippecanoe for preprocessing. Alternatively GDAL also has a gdal2tiles script if your input is already a raster. An open source option for frontend viewing is maplibre.

> [1] 2 min intro to 3D Gaussian splatting: https://www.youtube.com/watch?v=HVv_IQKlafQ

I don't know what the hell a "gaussian splat" is and I have to say this video did not help at all.

Here's a much better explainer from some people who seem to actually understand the subject matter and not just the buzzwords (and yes, the end result is basically a highly-detailed point cloud):

https://www.youtube.com/watch?v=VkIJbpdTujE

On mobile web you can't dismiss the text area. It covers 80% of the screen.

Thanks I just shrunk it a bunch more. It's a tech demo so apologies for the basic UI. It's also open source so anyone can remix it and play with the UI too