Development update

Paul

Thanks to all for patience on the current extended refactoring project on existing raster engine, now multi-domain, to support true multi-map and industrial-strength rapid scene switching/headless supporting design. This work has taken a whole month and still 2 more weeks likely until multimap.

Some side-projects happening in parallel are:

1) Client will have automatic notification circle when A) update is available, B) wire breaking update renders current build incompatible (forced update) or, C) when generic customizable operator message is set, which I will use to warn of wire breaking updates in advance. Screenshot of rollover tooltip text below.

2) A signature, defining UX element to WSV3 Tactical Mesoanalyst I have been envisioning for over a year: a slick, smooth pop/fade-in, progress-indicating, realtime-view-updating "Downlink" indicator and corresponding "signal capsule" native HUD UI element, showing the total download requirement for the given domains for the given camera view for the given loop length. The tiles per second indicator gives a baseline reference for raster data ingest speed that users can compare across machines and internet connections.

3) The realtime Websocket-based Tactical Server will collect the following connected client metadata/performance statistics, so I can monitor performance and quantify optimization results by collecting user GPU frame time, RAM, resolution, bandwidth use, and the like. Tactical Mesoanalyst adds value specifically to storm chasers and to mission critical in-field operational use, in power or bandwidth constrained environments, by being the most performance-optimized solution for something of its kind. So, I must track and obsess over these numbers.

Downlink indicator with "signal capsule" SDF widget:

Client stats:

Proper system-awareness and notification of wire-incompatible builds, using new SDF-based native D3D12 GPU rendered "signal capsule" widget:

Hovering update notification circle (tooltip background currently rendered behind black bar - glitch to be fixed):

Hovering operator message circle:

Paul

Recent email:

"My apologies for the delay. I can show you two new features that are side-work to my mainline complex, slow raster refactor for 8x multimap, 8x domain per layer, multi-layer design.

There is new very cool "timebar" native SDF Visor UI component, natively rendered in D3D12 with very slick 120fps expand/collapse animation effects (it looks iOS/Win11 native OS grade in terms of visual quality) - this is a multi-purpose visualizer for the frame load progress for all domains, showing time quantization / time selection, showing the individual domain frame timestamps on rollover, and also has a control/selection aspect: right now just click to drag/scrub timeline (you can hide the uglier full ImGui UI on the sidebars with H key), but soon it will have full temporal selection and ability to drag/slide the time window - a visual alternative to the basic controls on the ImGui Timeline UI dropdown.

The engine now supports forensic timelines - meaning you can loop to non-live, past end times. This means you can get full temporal resolution on historical periods within the history limit. Like, the base license gives a full 8hrs, but if you looped 8hrs of MESO domains, it has to quantize the frames at some point, or else it will donwload thousands of frames. Current timelines are 128 frames, I might bump that to 256. But the point is, you will be able to visually zoom into to the timebar to select a narrow time range, like a temporal equivalent of zooming into map spatially. So say a 2 hr window from 6 to 4 hrs ago, to see a supercell evolution with full 1-min MESO temporal resolution - this was not possible until now.

And cross-device automatic state/setting synchronization with instant Push and Pull buttons - this will synch the camera view, data selection, settings, customizations - everything.

Here are some screenshots. Unfortunately early May is still the best next update target. The raster engine work is very difficult. It's like 2 months of work just so that you can rapidily e.g. spam scrub the band selector (which has to issue an actual web message to server to change the "data scene") without even a single frame of blank-map-flicker appearing in between. This needs to be ultra fast and smooth for rapid interrogation and switching of content.

I will probably post this update to the forum."

JPWX

I'm really looking forward to the day radar is added

Paul

JPWX RadarHalo Polyradial Mosaic baseline availability defines the moment we can debut the CONUS license. July timeframe expected.

In the past week, I have attained great progress on the ingest component for RadarHalo. The next-gen WSV3 Tactical ingest server is now downloading the entire country NEXRAD L2 feed in realtime, with instant per-chunk event-based trigger, and transcoding all reflectivity chunks to the optimized fast-downloading format and uploading to server. This is shown in realtime on my admin console for the new mission-critical, bare-metal distributed system:

So the server-side work for RadarHalo Polyradial Mosaic is mostly done, leaving about a 1 month client-side rendering project. It's finally coming this summer.

donjgatlin

Paul, any chance you can prioritize adding at least US state boundaries? I know we're still in the early stages, but it does aid in matching up satellite to various landmarks. Great work.

Paul

donjgatlin Full-fledged server-hosted automatic-downloading CONUS-wide GIS/vector mapping engine with street + political boundary line tiles is coming immediately after the next major update. The next major update, mid-May, is debuting the final process and workflow for this product both for users and admin alike.

The engine will be shipped as an invisible background DLL to achieve the dream of automatic Chrome/Firefox-like updates. It can reload the entire engine and accomplish a whole update within 1-2 seconds, without the EXE ever reloading. This is a critical, novel development for the in-field/tactical purpose of no-bullshit fastest-humanly-possible user workflow for everything, not just loading data inside the program, but true workflow actions like initial load launch, subsequent launch, cross-device usage (instant sign-in with automatic other-PC live eviction), and now automatic updates and single-touch web-based cross device sync of settings/customizations/etc, in the next update.

But I would expect major progress on the GIS/vector mapping engine in the next 5 weeks. Yes, it's informationally difficult to lack this still. The vector mapping is critical to spatial perception.

Paul

I am working on finishing the holy grail of near-instant hassle-free updates. The engine is now fully modular and hot-reloadable within the same EXE without ever even closing the process. Updates to the host EXE will be rare. Regular updates will appear instantly as a top-right info icon, pushed immediately over Websocket just like new data frames, giving you the option to dismiss (auto apply at next EXE launch) or to apply the update live in realtime. It will should down and reload the engine, accomplishing the update within 1 or 2 seconds without the EXE ever closing.

Cold launch time is now massively optimized with parallel module downloads. It was under 300ms to engine init in some tests on residential internet.

I have also decided recently to commit to developing native Android build of Tactical Mesoanalyst, based on the Vulkan rendering engine already planned for the native Linux desktop version. I did not realize that Android device compatibility with modern high-performance Vulkan GPU API is now up to 85%. Given the extreme power optimization efforts and mobile/in-field/tactical use case which defines this product, and the intentions for a native Linux client anyways, it seems ridiculous not to go for the full 9 yards, Win32 x64 or ARM64 CPU, D3D12 or Vulkan for graphics, and native Linux Vulkan and native Android Vulkan.

Furthermore I don't envision selling this separately. You could use your same data license freely on any target, Windows desktop, Linux desktop, or Android mobile. If you buy, e.g., one $20/month/instance flagship CONUS data license, you should be able to bounce that seamlessly across your Android chasing tablet, home Linux desktop, or mobile Windows laptop, instantly, with the new VM-supporting floating licensing system. It enforces the concurrent usage limit without requiring you to re-enter keys. You paste your account ID once and then it remembers indefinitely. Instant cross-device usage.

With the settings/state/customization push/pull system shown above, you could configure your maps and styles and views on desktop, push, then instantly launch on Android, click pull, and everything is synched. This is a rapid workflow optimized for time sensitive mission-critical/storm-chasing contexts.

Win32 D3D12 remains the solely development target for the next few months, but I do hope to have the Vulkan rendering engine, the native POSIX compliant Linux port, and subsequently easy Android version, by end of year.

Target for next update is mid-late May.

Paul

Realtime-updating click drop solar info probe and live day/night shading effect, with distinct day/night map styles coming shortly. Also visible is a very low res initial scaffolding placeholder of the GIS/vector tile engine.

The noisy elevation-based fake coastline from the 0ft contour of elevation tiles will continue to look terrible, until the raster polygon cache variant is implemented, delivering accurate land/sea mask, coastline effect, and lakes.

Paul

I might able to email private new-system tests tonight. Either way, public release coming this week. GIS/vector engine line mapping work and raster engine multi-domain polish will be after this. There's so much else that's changed on client and server, I need to kill the old servers and cross everything over to the new scale-ready infrastructure.

GOES-R non-visible non-Band-7 bands (6, 8-16) will have a noticeably improved appearance/data quality in the new system, for about 2x the bandwidth, but they are already small compared to visible. The previous compression settings were causing artifacts. This is now the full resolution lossless compressed on IR/WV channels.

We also now have HDR swapchain automatic detection and support. If you have an OLED monitor and enable HDR in Windows settings, you will automatically see brighter whites and deeper contrast, adjustable in settings window.

There's also now fast, convenient Ctrl+S timestamped screenshot export with PNG or JPG outputs:

Paul

The cutover is happening now. Old servers will soon shut off and current clients will lose all data until updating to the upcoming release. I will manually send emails to streampack license holders. Going to try to keep this to 2 hour offline period. If you are online and see this message today, please shoot me an email so I can directly work with you to ensure the new system is up and running perfectly.

Paul

The new servers are deployed and new update live on WSV3.com/Tactical

I recommend deleting %LOCALAPPDATA%/WSV3 before running this.

The entire client is now built in pure C, having crossed-over from Pascal earlier this month. Pascal was equally low-level and performant, but the C complier infrastructure is better, and this decision will also make Linux/Android development somewhat easier.

The worst known bug in this current rushed release is an occasional graphics device lost crash to black screen, having to do with GPU fencing regressions on the probe stats readback. I will fix this soon. It wasn't happening before. Just relaunch in 3 seconds if it happens. It's rare enough it was not sensible to further delay this.

GOES-R derived products like Rainfall Rate Quantitative Precipitation Estimate (RRQPE) are now preliminarily included. They are raw grayscale only and not connected to palettes yet. Same for MRMS radar, which is now included temporarily along with G18/19 streampacks until we have the CONUS license.

I would like to give individual support over email with this build, or you can post here. Multidomain raster engine support is now live, though unpolished.

There is still no vector/GIS mapping at all. This update finally liberates me to focus on both the static raster background mapping and GIS/vector engine.

I don't currently have time to enumerate all the features and benefits from the last 3 months of work released just now, but you can scroll up to see post history above.

Soon I will like to try the first live operational update in the new system. This is where a small little icon in the right-top tray will appear. You click and it instantly reloads the engine, applying the updating in 2 or 3 seconds without the EXE ever closing.

Dynamic range palette is now default on Band 2. Ctrl+right mouse button zoom box and native GPU-rendered Visor timebar control are newly live as well:

Paul

Forgot to mention, the map pan/zoom and Visor buttons now have full touchscreen support. The two-finger combined drag/zoom camera gesture is extremely smooth and precise.

Paul

Baseline CONUS vector tiles achieved. In order to get this out fast, it downloads a temporary 4mb file that has all CONUS tiles. Next phase is to add dense road data where we will do the final form individually downloaded server-hosted vector tiles.

There are still some issues with the automatic update on launch using existing EXE, so I recommend redownloading the EXE to get this newest build. Perfecting the launch system and automatic updates is a goal for this week.

Paul

The next-gen server side ingest system is now processing, transcoding, and uploading the entire NEXRAD Level 2 dataset from all sites in realtime. This is about 10 TB per month and not cheap. This will enable a fast, rapid loading, bandwidth-optimized, download-what-I'm-looking-at raw L2 viewing experience like how RadarScope offers.

I can't state enough how terrible the Gibson Ridge/WSV3 2015 model is in this regard, compared to the RadarScope/WSV3 Tactical model where the company has to pay to process the live feed and pay to disseminate it to their proprietary viewer app, instead of pointing the client at the huge raw L2 volumes.

The level of waste in the latter is insane. It could be 100x wasted bandwidth if you're just viewing L2 base refl.

We are now unblocked on NEXRAD/RadarHalo Polyradial Mosaic client side development. CONUS license target remains late July.

Another decision is to prioritize 3D in place of multimap. 3D has been so fundamental to this next-gen product that I'd like it to be in the baseline $20/month license. Possibly limited to 4x multimap with 8x only in 2x Workstation license. I'd rather do that than confine 3D. The basic $20/month CONUS standard experience needs to have 3D and multimap. I think saving 8x for the Workstation license is a fair differentiation.

3D is no longer an August/September project. I haven't seen my stunningly compelling October 2023 - summer 2024 3D terrain engine since mid-July 2024. We all miss those graphics.

I want it back for July and am now actively preparing for this.

Paul

From email:

"P.S. - I forgot the MRMS screenshot attachment. MRMS is derived data anyway, not raw sensor (like L2 or GOES-R ABI) so I had no problem downscaling and filtering this to make it look nice. Like first-gen it's 1/2 downscaled, which makes it look better since the MRMS resampling is awful, but this time, the ingest server applies a 4-tap 2D Gaussian filter, supports negative reflectivities at half steps to use the full 8-bit range (-27.5dBz to +100dBz), and retains the native 2-minute cadence. First-gen uses every 3rd frame (6mins).

I am very excited to really bring this alive with the finished vector/GIS mapping engine which will take a few weeks. It will start with dense national-scale local street mapping and road data, with server-hosted tiles, then embrace the whole labeling engine aspect, with server-hosted tiles for city/place names and angled street rendering. It will be incredible.

Also decided to prioritize 3D over multimap. I now plan to release the CONUS license late July with basic 3D (camera controls, flat-sphere no-terrain only, atmosphere shading, and full raster + vector engine support) but without multimap. I now think this is better than doing multimap first or further delaying the CONUS license."

Paul

New update with the above just released, and with attempted fixes of the auto-update launch system. Redownload EXE from site is necessary again but launch updates should be stable now.

OregonCoastWx

Not sure if I downloaded the exe file for this latest update, but i don't see no MRMS radar? I used this site too download, https://wsv3.com/Tactical/ unless i misunderstood that this is operational now.

Paul

OregonCoastWx The data is paid; please email me for a 24-hour complimentary live data test account so you can check out the progress. If you'd benefit from raw resolution multidomain satellite imagery, the $20/month GOES-19 Streampack has been avaliable at WSV3.com/Tactical since December, but otherwise, the CONUS license with radar, warnings, and select satellite imagery domains is planned for late July release.

OregonCoastWx

Ok Paul, I have had this paid data since around March 1st and I just misunderstood the timeline, keep up the awesome work.

Paul

OregonCoastWx If you have a GOES-19 streampack currently in Tactical, you should be able to access the placeholder MRMS radar data. See the selector on top of Data Source tab, "GOES-R Imagery" by default. Click that to MRMS.

« Previous Page Next Page »