carverauto/serviceradar

Fork 0

feat(web-ng): topo map improvements #1061

New issue

Open

opened 2026-03-28 04:31:18 +00:00 by mfreeman451 · 1 comment

mfreeman451 commented

2026-03-28 04:31:18 +00:00

Owner

Imported from GitHub.

Original GitHub issue: #2912
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/2912
Original created: 2026-02-25T12:53:08Z

PRD: Unified Topology Investigation Surface

Product: ServiceRadar
Author: Carver Automation Corporation
Status: Draft
Created: 2025-02-25
Rendering Stack: deck.gl / luma.gl

1. Problem Statement

Network operators and SOC analysts are forced to context-switch between siloed tools — NMS platforms for device health and topology, SIEM consoles for security events, flow analyzers for traffic forensics, and vulnerability scanners for posture assessment. During an active incident, this fragmentation costs critical minutes as analysts manually correlate data across systems to answer fundamental questions:

"What was this device talking to at 2:14 AM when the alert fired?"
"Which physical path did the malicious traffic traverse?"
"Is the host with the container breakout also talking to a known-bad IP?"
"Did the BGP route flap cause the traffic shift, or was it something else?"

ServiceRadar already ingests the signals needed to answer these questions — NetFlow, SNMP, syslog, BGP/BMP, threat feeds (AlienVault OTX), Falco/Trivy, and GeoIP/ASN enrichment. The topology surface already renders device relationships using deck.gl. The opportunity is to fuse these signals into a single, high-performance visual investigation plane where the topology graph becomes the primary entry point for both network troubleshooting and security investigation.

2. Vision

The ServiceRadar Topology Surface becomes a unified investigation canvas — the place where a network operator troubleshooting a routing issue and a SOC analyst investigating lateral movement both start their work. Every node on the graph is a portal to correlated, multi-signal context. Every edge encodes real-time network state. The graph itself becomes a living anomaly map that surfaces problems before analysts go looking for them.

3. Goals & Success Metrics

Goal	Metric
Reduce mean-time-to-identify (MTTI) for network incidents	Analyst can go from topology view to root cause hypothesis in < 3 clicks
Eliminate NMS ↔ SIEM context-switching	Zero external tool pivots required for first-pass triage
Support real-time investigation at scale	60fps rendering with 50,000+ nodes and active NetFlow overlays
Surface threats proactively on the topology	Time from threat feed match to visual indicator on graph < 30 seconds

4. User Personas

Network Operator (NetOps)

Responsible for uptime, performance, and troubleshooting. Cares about link utilization, device health, routing state, and traffic patterns. Needs to quickly answer: "What changed and where is the bottleneck?"

SOC Analyst

Responsible for threat detection, investigation, and response. Cares about anomalous communications, known-bad IPs, vulnerability exposure, and lateral movement. Needs to quickly answer: "What is the blast radius and how did the attacker move?"

Network Security Engineer

Straddles both worlds. Maintains firewall policies, segments networks, monitors for policy violations. Needs the topology to show both logical security boundaries and the traffic that crosses them.

5. Feature Specifications

5.1 NetFlow Conversation Arcs (ArcLayer)

Trigger: User clicks a node, or a threat-correlated flow is detected.

Behavior:

On node selection, query NetFlow records where the selected node's IP appears as source or destination within the active time window.
Dim background topology edges (reduce opacity to ~20%).
Render a deck.gl ArcLayer from the selected node to every peer it communicated with.
For peers that exist on the topology (internal), arcs connect directly between nodes.
For external peers, arcs terminate at dynamically generated peripheral nodes (see §5.2).

Visual Encoding:

Property	Maps To	Example
Arc width	Bytes transferred (log scale)	Thick arc = high-volume flow
Arc color	Protocol / traffic class	Blue = HTTPS, Yellow = DNS, Red = SSH/Telnet, Magenta = flagged
Arc animation	Flow direction (src → dst)	luma.gl shader pulse traveling along arc
Arc opacity	Recency	Recent flows opaque, older flows fade

Interaction:

Hover over an arc to see a tooltip with: source IP, destination IP, protocol, port, bytes, packets, flow start/end time, and ASN/GeoIP for external peers.
Click an arc to pin it and open the flow detail in the context panel (§5.7).

Performance Guard: NetFlow arcs render only for the selected node or for threat-correlated flows. Global "show all flows" is intentionally omitted to prevent visual overload (the "hairball" problem). Use deck.gl Composite Layers to keep the arc layer separate and conditionally instantiated.

5.2 External Traffic & GeoIP Orbit

Problem: NetFlow conversations with external IPs have no node on the LAN topology to terminate at.

Solution: Generate a dynamic external orbit — a ring of clustered nodes positioned along the periphery of the viewport, grouped by ASN or geographic region.

Behavior:

When NetFlow arcs to external IPs are rendered, cluster destination IPs by ASN (primary) or country (fallback).
Render each cluster as a labeled node on the periphery (e.g., "AS16509 — Amazon", "AS13335 — Cloudflare", "RU — Russia").
Arcs from the selected internal node extend to these peripheral clusters.
Threat-flagged clusters (any IP in the cluster matches AlienVault OTX) render with a red halo.

Future Extension: Toggle to a globe/map view (deck.gl GlobeView) where external arcs land on geographic coordinates, using GeoIP enrichment for placement.

5.3 Threat & Vulnerability Overlays

These overlays transform the topology from a network diagram into a security posture map.

5.3.1 Threat Feed Correlation (AlienVault OTX)

Continuously correlate incoming NetFlow against the active threat feed.
When a match is found:
- The internal node(s) that communicated with the flagged IP receive a pulsing red halo (animated glow ring around the node's scatterplot point).
- If the physical path is known (via topology edges), render a PathLayer with a neon red/orange glow (luma.gl bloom post-processing) tracing the traffic path through intermediate switches/routers.
- Auto-generate an event in the timeline (§5.5).
Click the halo to open the context panel showing: the matched indicator (IP, domain, hash), OTX pulse details, the NetFlow session(s) involved, and the physical path traversed.

5.3.2 Falco / Trivy Badges

If a topology node represents a host running containers:
- Trivy: Overlay a shield icon badge with a count of critical/high CVEs. Color: orange (high), red (critical).
- Falco: On active alert, pulse the node's center dot from white to crimson. Badge with alert count.
Click the badge to expand Falco events and Trivy findings in the context panel.

5.3.3 SNMP Trap & Syslog Indicators

Nodes with unacknowledged SNMP traps or high-severity syslog events display a small warning triangle badge.
Severity-mapped: yellow (warning), orange (error), red (critical).

5.4 SNMP Metric Encoding on Edges

Replace static topology edges with data-driven visual links reflecting real-time interface metrics.

Link Saturation Color Ramp:

Utilization	Color
0–50%	Cool blue (current default aesthetic)
50–75%	Yellow
75–90%	Orange
90–100%	Red

Particle Encoding:

Bind the existing particle animation speed and density to actual PPS (packets per second) or flow record counts from SNMP polling.
A DDoS or broadcast storm becomes immediately visible as an overwhelming flood of fast-moving particles on a specific trunk link, without the analyst needing to check a dashboard.

Link State:

Interface down: edge rendered as dashed line with reduced opacity.
Flapping: edge pulses between solid and dashed.

5.5 Temporal Investigation ("Time-Travel DVR")

Purpose: Enable analysts to answer "What was the network doing at time X?"

UI Element: A timeline scrubber bar at the bottom of the topology surface.

Behavior:

Default position: Live (real-time data).
Drag the scrubber to a historical timestamp.
All visual layers update to reflect network state at that time:
- Node health colors reflect SNMP status at that timestamp.
- If NetFlow arcs are active, they reflect flows from the selected time window.
- Threat overlays show which indicators were active.
- SNMP edge utilization colors update.
Playback controls: play/pause, speed (1x, 5x, 10x, 60x), step forward/back by configurable interval.
Bookmarking: Analysts can pin timestamps to the timeline (e.g., "Alert fired here", "Traffic spike started here") during investigations.

Data Source: TimescaleDB continuous aggregates for SNMP metrics; NetFlow stored in time-bucketed hypertables; event timestamps from syslog/Falco/traps.

5.6 Topology View Modes

Provide toggleable views to let analysts switch the lens through which they see the network.

Mode	Layer Source	Edge Semantics
Physical / L2 (default)	CDP/LLDP discovery, SNMP	Physical connections, switch-to-switch trunks
L3 Routing	Routing table polling, ARP	IP adjacencies, subnet boundaries
BGP Peering	BMP (BGP Monitoring Protocol)	BGP sessions between peers
NetFlow Logical	NetFlow/IPFIX	Traffic conversations (ArcLayer only, no physical edges)

BGP-Specific Behaviors:

BGP session flap: arc snaps, flashes yellow, syslog events populate in the live feed panel.
Route withdrawal/announcement: briefly highlight affected prefixes on the nodes that advertise them.
Hijack detection (unexpected origin AS for a prefix): red arc + alert badge.

5.7 Unified Context Panel

Trigger: Click any node or edge on the topology.

Layout: Slide-out drawer from the right side of the viewport (does not obscure the topology; topology shifts left).

Content — Node Selected:

Section	Data Source
Device Summary	Hostname, IP(s), MAC, model, uptime, location
Interface Table	SNMP — per-interface utilization, errors, status
Top Talkers	NetFlow — top N peers by bytes, with protocol breakdown
Recent Events	Syslog + SNMP traps — chronological feed, severity-colored
Security Posture	Falco alerts, Trivy CVE summary, threat feed matches
Flow Timeline	Sparkline of total traffic volume over the active time window

Content — Edge Selected:

Section	Data Source
Link Summary	Endpoints, interface names, speed, VLAN(s)
Utilization Chart	SNMP — mini time-series chart (sparkline or Recharts)
Top Applications	NetFlow — top N protocols/applications on this link
Error Counters	SNMP — CRC errors, input/output errors, discards

5.8 Investigation Mode

A dedicated workflow for active incident response and threat hunting.

5.8.1 Right-Click → Investigate

Right-clicking a node opens a context menu with "Investigate" as the primary action.
Opens the context panel (§5.7) pre-loaded with all signal sources scoped to a configurable time window (default: last 1 hour).
All signals are interleaved chronologically in a unified event timeline within the panel.
Each event in the timeline is clickable — NetFlow sessions link to their peer node on the topology, syslog entries highlight the originating device, etc.

5.8.2 Path Trace

Analyst selects two nodes (shift-click or via a "Trace Path" dialog).
The topology highlights all L2/L3 paths between them.
NetFlow data for conversations between those two endpoints overlays as arcs.
Each intermediate hop shows its interface utilization and any events that occurred during the time window.

5.8.3 SRQL Integration

A command bar (hotkey: /) allows analysts to type SRQL queries that filter and highlight the topology.
Example queries:
- flows where dst_asn = 'AS1234' and bytes > 1GB last 24h — highlights all nodes involved in matching flows, renders arcs.
- events where severity = 'critical' last 6h — pulses all nodes with matching events.
- devices where trivy_critical > 0 — highlights vulnerable hosts.
The topology acts as a query result renderer — SRQL output maps back onto the spatial graph.

5.8.4 Anomaly Overlays

Baseline analysis (powered by TimescaleDB continuous aggregates) flags statistical anomalies:
- Unusual traffic volume to/from a device.
- New destination ASNs not seen in the baseline period.
- Protocol distribution shifts (e.g., sudden spike in DNS or SSH).
- New device-to-device conversations not previously observed.
Anomalies surface as subtle animated indicators on the affected nodes/edges (e.g., a slowly pulsing amber ring).
Clicking the indicator shows the anomaly detail: what changed, baseline vs. current values, when it started.

5.9 Anti-Hairball Rendering Strategy

At scale (50,000+ nodes), visual clarity is paramount. The following strategies prevent the topology from becoming an unreadable mess.

Strategy	Implementation
Lazy arc instantiation	NetFlow ArcLayer data only loads on explicit trigger (node click, threat match, SRQL query). Never render all flows globally.
Semantic zoom	Zoomed out: aggregate to subnet or site-level clusters with inter-cluster flow arcs. Zoomed in: individual device nodes and per-device flows.
Layer compositing	Each visual concern is a separate deck.gl Composite Layer that can be independently toggled: base topology, SNMP heatmap, NetFlow arcs, threat overlays, anomaly indicators.
Viewport culling	Only render arcs/overlays for nodes currently visible in the viewport. Defer off-screen data.
Progressive disclosure	Default state is clean topology with SNMP edge coloring. Threat halos and anomaly indicators appear passively. All other overlays require explicit user action.

6. Data Flow Architecture

                    ┌─────────────────────────────────────────────────┐
                    │              Signal Ingestion                    │
                    │                                                  │
                    │  NetFlow/IPFIX  SNMP Polls  Syslog  BGP/BMP    │
                    │  SNMP Traps     Falco       Trivy   OTX Feed   │
                    └──────────────────────┬──────────────────────────┘
                                           │
                                           ▼
                    ┌─────────────────────────────────────────────────┐
                    │           Enrichment Pipeline                   │
                    │                                                 │
                    │    • GeoIP / ASN lookup on NetFlow              │
                    │    • OTX correlation on dst IPs                 │
                    │    • Device resolution (IP → topology node)     │
                    └──────────────────────┬──────────────────────────┘
                                           │
                              ┌─────────────┴─────────────┐
                              ▼                           ▼
                    ┌──────────────────┐        ┌──────────────────┐
                    │   TimescaleDB    │        │   Event Store    │
                    │                  │        │                  │
                    │  • Flow records  │        │  • Syslog events │
                    │  • SNMP metrics  │        │  • SNMP traps    │
                    │  • Continuous    │        │  • Falco alerts  │
                    │    aggregates    │        │  • Threat matches│
                    │  • Baselines     │        │  • BGP events    │
                    └────────┬─────────┘        └────────┬─────────┘
                             │                           │
                             └─────────────┬─────────────┘
                                           │
                                           ▼
                    ┌─────────────────────────────────────────────────┐
                    │              API / Query Layer                   │
                    │                                                  │
                    │  gRPC Services          SRQL Translator          │
                    │  (topology, flows,      (SRQL → SQL, returns    │
                    │   events, metrics)       node/edge highlights)  │
                    │                                                  │
                    │  MCP Interface                                   │
                    │  (LLM-friendly access for AI-assisted triage)   │
                    └──────────────────────┬──────────────────────────┘
                                           │
                                           ▼
                    ┌─────────────────────────────────────────────────┐
                    │         Topology Investigation Surface           │
                    │                                                  │
                    │  deck.gl Layer Stack:                            │
                    │    ScatterplotLayer ─── Nodes                    │
                    │    LineLayer ────────── Base topology edges      │
                    │    ArcLayer ─────────── NetFlow conversations    │
                    │    PathLayer ────────── Threat traffic paths     │
                    │    IconLayer ────────── Alert/CVE badges         │
                    │    TextLayer ────────── Labels                   │
                    │                                                  │
                    │  + Context Panel, Timeline, SRQL Bar, Controls  │
                    └─────────────────────────────────────────────────┘

7. Implementation Phases

Phase 1 — Foundation (NetFlow Arcs + Context Panel)

Implement node-click → NetFlow ArcLayer with protocol color encoding and width scaling.
Build the unified context panel (§5.7) with device summary, top talkers, and recent events.
Wire SNMP utilization color ramp onto existing topology edges (§5.4).
Add hover tooltips on arcs and edges with summary data.

Phase 2 — Security Overlays

Integrate AlienVault OTX correlation into the enrichment pipeline; render threat halos on nodes (§5.3.1).
Add Falco/Trivy badge overlays (§5.3.2).
Implement the external orbit for GeoIP/ASN-clustered external peers (§5.2).
SNMP trap and syslog warning badges (§5.3.3).

Phase 3 — Temporal & Investigation

Build the time-travel DVR with timeline scrubber (§5.5).
Implement Investigation Mode: right-click investigate, path trace, unified event timeline (§5.8.1, §5.8.2).
Wire SRQL queries to topology highlighting (§5.8.3).

Phase 4 — Intelligence & Scale

Anomaly detection overlays using TimescaleDB baseline analysis (§5.8.4).
Semantic zoom with subnet/site-level aggregation for 50k+ node deployments (§5.9).
BGP/BMP view mode with session state visualization (§5.6).
Globe/map view toggle for external traffic geolocation.

8. Open Questions

#	Question	Impact
1	Should NetFlow arc animation use `TripsLayer` (built-in) or custom luma.gl shaders?	Phase 1 — rendering approach and performance envelope
2	How granular should the time-travel DVR be? Per-minute? Per-5-minutes?	Phase 3 — storage costs and query performance in TimescaleDB
3	Should SRQL topology highlighting happen client-side (filter rendered data) or server-side (API returns node/edge IDs to highlight)?	Phase 3 — architecture of SRQL ↔ topology integration
4	What is the aggregation strategy for semantic zoom? Subnet-based? Site/location-based? User-defined groups?	Phase 4 — UX and data modeling
5	Should the MCP interface support natural-language investigation queries (e.g., "show me all traffic to Russia in the last 24 hours") that translate to SRQL + topology highlights?	Phase 4 — AI-assisted triage workflow
6	Threat path rendering (§5.3.1) requires mapping NetFlow 5-tuples to physical topology hops. Is the current topology model sufficient, or do we need explicit L2 path tracing?	Phase 2 — data model completeness

9. Competitive Differentiation

Most NMS tools (LibreNMS, Zabbix, PRTG) render topology with link utilization overlays but have no security signal integration. Most SIEMs (Splunk, Elastic SIEM, Wazuh) present log events in tables and timelines but have no network topology awareness. The small number of tools that attempt both (SolarWinds, Cisco DNA Center) are proprietary, expensive, and architecturally siloed internally.

ServiceRadar's approach — a single deck.gl-powered canvas that spatially correlates network state, traffic flows, and security signals — is architecturally novel in the open-source space. The combination of SRQL as a query language, MCP for AI-assisted triage, and the real-time rendering performance of deck.gl/luma.gl positions ServiceRadar to be the first open-source platform that genuinely unifies NMS and SIEM investigation workflows on a shared visual surface.

Imported from GitHub. Original GitHub issue: #2912 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/2912 Original created: 2026-02-25T12:53:08Z --- # PRD: Unified Topology Investigation Surface **Product:** ServiceRadar **Author:** Carver Automation Corporation **Status:** Draft **Created:** 2025-02-25 **Rendering Stack:** deck.gl / luma.gl --- ## 1. Problem Statement Network operators and SOC analysts are forced to context-switch between siloed tools — NMS platforms for device health and topology, SIEM consoles for security events, flow analyzers for traffic forensics, and vulnerability scanners for posture assessment. During an active incident, this fragmentation costs critical minutes as analysts manually correlate data across systems to answer fundamental questions: - *"What was this device talking to at 2:14 AM when the alert fired?"* - *"Which physical path did the malicious traffic traverse?"* - *"Is the host with the container breakout also talking to a known-bad IP?"* - *"Did the BGP route flap cause the traffic shift, or was it something else?"* ServiceRadar already ingests the signals needed to answer these questions — NetFlow, SNMP, syslog, BGP/BMP, threat feeds (AlienVault OTX), Falco/Trivy, and GeoIP/ASN enrichment. The topology surface already renders device relationships using deck.gl. The opportunity is to fuse these signals into a single, high-performance visual investigation plane where the topology graph becomes the primary entry point for both network troubleshooting and security investigation. --- ## 2. Vision The ServiceRadar Topology Surface becomes a **unified investigation canvas** — the place where a network operator troubleshooting a routing issue and a SOC analyst investigating lateral movement both start their work. Every node on the graph is a portal to correlated, multi-signal context. Every edge encodes real-time network state. The graph itself becomes a living anomaly map that surfaces problems before analysts go looking for them. --- ## 3. Goals & Success Metrics | Goal | Metric | |------|--------| | Reduce mean-time-to-identify (MTTI) for network incidents | Analyst can go from topology view to root cause hypothesis in < 3 clicks | | Eliminate NMS ↔ SIEM context-switching | Zero external tool pivots required for first-pass triage | | Support real-time investigation at scale | 60fps rendering with 50,000+ nodes and active NetFlow overlays | | Surface threats proactively on the topology | Time from threat feed match to visual indicator on graph < 30 seconds | --- ## 4. User Personas ### Network Operator (NetOps) Responsible for uptime, performance, and troubleshooting. Cares about link utilization, device health, routing state, and traffic patterns. Needs to quickly answer: *"What changed and where is the bottleneck?"* ### SOC Analyst Responsible for threat detection, investigation, and response. Cares about anomalous communications, known-bad IPs, vulnerability exposure, and lateral movement. Needs to quickly answer: *"What is the blast radius and how did the attacker move?"* ### Network Security Engineer Straddles both worlds. Maintains firewall policies, segments networks, monitors for policy violations. Needs the topology to show both logical security boundaries and the traffic that crosses them. --- ## 5. Feature Specifications ### 5.1 NetFlow Conversation Arcs (ArcLayer) **Trigger:** User clicks a node, or a threat-correlated flow is detected. **Behavior:** - On node selection, query NetFlow records where the selected node's IP appears as source or destination within the active time window. - Dim background topology edges (reduce opacity to ~20%). - Render a `deck.gl ArcLayer` from the selected node to every peer it communicated with. - For peers that exist on the topology (internal), arcs connect directly between nodes. - For external peers, arcs terminate at dynamically generated peripheral nodes (see §5.2). **Visual Encoding:** | Property | Maps To | Example | |----------|---------|---------| | Arc width | Bytes transferred (log scale) | Thick arc = high-volume flow | | Arc color | Protocol / traffic class | Blue = HTTPS, Yellow = DNS, Red = SSH/Telnet, Magenta = flagged | | Arc animation | Flow direction (src → dst) | luma.gl shader pulse traveling along arc | | Arc opacity | Recency | Recent flows opaque, older flows fade | **Interaction:** - Hover over an arc to see a tooltip with: source IP, destination IP, protocol, port, bytes, packets, flow start/end time, and ASN/GeoIP for external peers. - Click an arc to pin it and open the flow detail in the context panel (§5.7). **Performance Guard:** NetFlow arcs render only for the selected node or for threat-correlated flows. Global "show all flows" is intentionally omitted to prevent visual overload (the "hairball" problem). Use deck.gl Composite Layers to keep the arc layer separate and conditionally instantiated. --- ### 5.2 External Traffic & GeoIP Orbit **Problem:** NetFlow conversations with external IPs have no node on the LAN topology to terminate at. **Solution:** Generate a dynamic **external orbit** — a ring of clustered nodes positioned along the periphery of the viewport, grouped by ASN or geographic region. **Behavior:** - When NetFlow arcs to external IPs are rendered, cluster destination IPs by ASN (primary) or country (fallback). - Render each cluster as a labeled node on the periphery (e.g., "AS16509 — Amazon", "AS13335 — Cloudflare", "RU — Russia"). - Arcs from the selected internal node extend to these peripheral clusters. - Threat-flagged clusters (any IP in the cluster matches AlienVault OTX) render with a red halo. **Future Extension:** Toggle to a globe/map view (deck.gl `GlobeView`) where external arcs land on geographic coordinates, using GeoIP enrichment for placement. --- ### 5.3 Threat & Vulnerability Overlays These overlays transform the topology from a network diagram into a **security posture map**. #### 5.3.1 Threat Feed Correlation (AlienVault OTX) - Continuously correlate incoming NetFlow against the active threat feed. - When a match is found: - The internal node(s) that communicated with the flagged IP receive a **pulsing red halo** (animated glow ring around the node's scatterplot point). - If the physical path is known (via topology edges), render a `PathLayer` with a neon red/orange glow (luma.gl bloom post-processing) tracing the traffic path through intermediate switches/routers. - Auto-generate an event in the timeline (§5.5). - **Click the halo** to open the context panel showing: the matched indicator (IP, domain, hash), OTX pulse details, the NetFlow session(s) involved, and the physical path traversed. #### 5.3.2 Falco / Trivy Badges - If a topology node represents a host running containers: - **Trivy:** Overlay a shield icon badge with a count of critical/high CVEs. Color: orange (high), red (critical). - **Falco:** On active alert, pulse the node's center dot from white to crimson. Badge with alert count. - Click the badge to expand Falco events and Trivy findings in the context panel. #### 5.3.3 SNMP Trap & Syslog Indicators - Nodes with unacknowledged SNMP traps or high-severity syslog events display a small warning triangle badge. - Severity-mapped: yellow (warning), orange (error), red (critical). --- ### 5.4 SNMP Metric Encoding on Edges Replace static topology edges with **data-driven visual links** reflecting real-time interface metrics. **Link Saturation Color Ramp:** | Utilization | Color | |-------------|-------| | 0–50% | Cool blue (current default aesthetic) | | 50–75% | Yellow | | 75–90% | Orange | | 90–100% | Red | **Particle Encoding:** - Bind the existing particle animation speed and density to actual PPS (packets per second) or flow record counts from SNMP polling. - A DDoS or broadcast storm becomes immediately visible as an overwhelming flood of fast-moving particles on a specific trunk link, without the analyst needing to check a dashboard. **Link State:** - Interface down: edge rendered as dashed line with reduced opacity. - Flapping: edge pulses between solid and dashed. --- ### 5.5 Temporal Investigation ("Time-Travel DVR") **Purpose:** Enable analysts to answer *"What was the network doing at time X?"* **UI Element:** A timeline scrubber bar at the bottom of the topology surface. **Behavior:** - Default position: **Live** (real-time data). - Drag the scrubber to a historical timestamp. - All visual layers update to reflect network state at that time: - Node health colors reflect SNMP status at that timestamp. - If NetFlow arcs are active, they reflect flows from the selected time window. - Threat overlays show which indicators were active. - SNMP edge utilization colors update. - Playback controls: play/pause, speed (1x, 5x, 10x, 60x), step forward/back by configurable interval. - **Bookmarking:** Analysts can pin timestamps to the timeline (e.g., "Alert fired here", "Traffic spike started here") during investigations. **Data Source:** TimescaleDB continuous aggregates for SNMP metrics; NetFlow stored in time-bucketed hypertables; event timestamps from syslog/Falco/traps. --- ### 5.6 Topology View Modes Provide toggleable views to let analysts switch the lens through which they see the network. | Mode | Layer Source | Edge Semantics | |------|-------------|----------------| | **Physical / L2** (default) | CDP/LLDP discovery, SNMP | Physical connections, switch-to-switch trunks | | **L3 Routing** | Routing table polling, ARP | IP adjacencies, subnet boundaries | | **BGP Peering** | BMP (BGP Monitoring Protocol) | BGP sessions between peers | | **NetFlow Logical** | NetFlow/IPFIX | Traffic conversations (ArcLayer only, no physical edges) | **BGP-Specific Behaviors:** - BGP session flap: arc snaps, flashes yellow, syslog events populate in the live feed panel. - Route withdrawal/announcement: briefly highlight affected prefixes on the nodes that advertise them. - Hijack detection (unexpected origin AS for a prefix): red arc + alert badge. --- ### 5.7 Unified Context Panel **Trigger:** Click any node or edge on the topology. **Layout:** Slide-out drawer from the right side of the viewport (does not obscure the topology; topology shifts left). **Content — Node Selected:** | Section | Data Source | |---------|-------------| | Device Summary | Hostname, IP(s), MAC, model, uptime, location | | Interface Table | SNMP — per-interface utilization, errors, status | | Top Talkers | NetFlow — top N peers by bytes, with protocol breakdown | | Recent Events | Syslog + SNMP traps — chronological feed, severity-colored | | Security Posture | Falco alerts, Trivy CVE summary, threat feed matches | | Flow Timeline | Sparkline of total traffic volume over the active time window | **Content — Edge Selected:** | Section | Data Source | |---------|-------------| | Link Summary | Endpoints, interface names, speed, VLAN(s) | | Utilization Chart | SNMP — mini time-series chart (sparkline or Recharts) | | Top Applications | NetFlow — top N protocols/applications on this link | | Error Counters | SNMP — CRC errors, input/output errors, discards | --- ### 5.8 Investigation Mode A dedicated workflow for active incident response and threat hunting. #### 5.8.1 Right-Click → Investigate - Right-clicking a node opens a context menu with "Investigate" as the primary action. - Opens the context panel (§5.7) pre-loaded with all signal sources scoped to a configurable time window (default: last 1 hour). - All signals are interleaved chronologically in a unified event timeline within the panel. - Each event in the timeline is clickable — NetFlow sessions link to their peer node on the topology, syslog entries highlight the originating device, etc. #### 5.8.2 Path Trace - Analyst selects two nodes (shift-click or via a "Trace Path" dialog). - The topology highlights all L2/L3 paths between them. - NetFlow data for conversations between those two endpoints overlays as arcs. - Each intermediate hop shows its interface utilization and any events that occurred during the time window. #### 5.8.3 SRQL Integration - A command bar (hotkey: `/`) allows analysts to type SRQL queries that filter and highlight the topology. - Example queries: - `flows where dst_asn = 'AS1234' and bytes > 1GB last 24h` — highlights all nodes involved in matching flows, renders arcs. - `events where severity = 'critical' last 6h` — pulses all nodes with matching events. - `devices where trivy_critical > 0` — highlights vulnerable hosts. - The topology acts as a **query result renderer** — SRQL output maps back onto the spatial graph. #### 5.8.4 Anomaly Overlays - Baseline analysis (powered by TimescaleDB continuous aggregates) flags statistical anomalies: - Unusual traffic volume to/from a device. - New destination ASNs not seen in the baseline period. - Protocol distribution shifts (e.g., sudden spike in DNS or SSH). - New device-to-device conversations not previously observed. - Anomalies surface as subtle animated indicators on the affected nodes/edges (e.g., a slowly pulsing amber ring). - Clicking the indicator shows the anomaly detail: what changed, baseline vs. current values, when it started. --- ### 5.9 Anti-Hairball Rendering Strategy At scale (50,000+ nodes), visual clarity is paramount. The following strategies prevent the topology from becoming an unreadable mess. | Strategy | Implementation | |----------|----------------| | **Lazy arc instantiation** | NetFlow ArcLayer data only loads on explicit trigger (node click, threat match, SRQL query). Never render all flows globally. | | **Semantic zoom** | Zoomed out: aggregate to subnet or site-level clusters with inter-cluster flow arcs. Zoomed in: individual device nodes and per-device flows. | | **Layer compositing** | Each visual concern is a separate deck.gl Composite Layer that can be independently toggled: base topology, SNMP heatmap, NetFlow arcs, threat overlays, anomaly indicators. | | **Viewport culling** | Only render arcs/overlays for nodes currently visible in the viewport. Defer off-screen data. | | **Progressive disclosure** | Default state is clean topology with SNMP edge coloring. Threat halos and anomaly indicators appear passively. All other overlays require explicit user action. | --- ## 6. Data Flow Architecture ``` ┌─────────────────────────────────────────────────┐ │ Signal Ingestion │ │ │ │ NetFlow/IPFIX SNMP Polls Syslog BGP/BMP │ │ SNMP Traps Falco Trivy OTX Feed │ └──────────────────────┬──────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────┐ │ Enrichment Pipeline │ │ │ │ • GeoIP / ASN lookup on NetFlow │ │ • OTX correlation on dst IPs │ │ • Device resolution (IP → topology node) │ └──────────────────────┬──────────────────────────┘ │ ┌─────────────┴─────────────┐ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ │ TimescaleDB │ │ Event Store │ │ │ │ │ │ • Flow records │ │ • Syslog events │ │ • SNMP metrics │ │ • SNMP traps │ │ • Continuous │ │ • Falco alerts │ │ aggregates │ │ • Threat matches│ │ • Baselines │ │ • BGP events │ └────────┬─────────┘ └────────┬─────────┘ │ │ └─────────────┬─────────────┘ │ ▼ ┌─────────────────────────────────────────────────┐ │ API / Query Layer │ │ │ │ gRPC Services SRQL Translator │ │ (topology, flows, (SRQL → SQL, returns │ │ events, metrics) node/edge highlights) │ │ │ │ MCP Interface │ │ (LLM-friendly access for AI-assisted triage) │ └──────────────────────┬──────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────┐ │ Topology Investigation Surface │ │ │ │ deck.gl Layer Stack: │ │ ScatterplotLayer ─── Nodes │ │ LineLayer ────────── Base topology edges │ │ ArcLayer ─────────── NetFlow conversations │ │ PathLayer ────────── Threat traffic paths │ │ IconLayer ────────── Alert/CVE badges │ │ TextLayer ────────── Labels │ │ │ │ + Context Panel, Timeline, SRQL Bar, Controls │ └─────────────────────────────────────────────────┘ ``` --- ## 7. Implementation Phases ### Phase 1 — Foundation (NetFlow Arcs + Context Panel) - Implement node-click → NetFlow ArcLayer with protocol color encoding and width scaling. - Build the unified context panel (§5.7) with device summary, top talkers, and recent events. - Wire SNMP utilization color ramp onto existing topology edges (§5.4). - Add hover tooltips on arcs and edges with summary data. ### Phase 2 — Security Overlays - Integrate AlienVault OTX correlation into the enrichment pipeline; render threat halos on nodes (§5.3.1). - Add Falco/Trivy badge overlays (§5.3.2). - Implement the external orbit for GeoIP/ASN-clustered external peers (§5.2). - SNMP trap and syslog warning badges (§5.3.3). ### Phase 3 — Temporal & Investigation - Build the time-travel DVR with timeline scrubber (§5.5). - Implement Investigation Mode: right-click investigate, path trace, unified event timeline (§5.8.1, §5.8.2). - Wire SRQL queries to topology highlighting (§5.8.3). ### Phase 4 — Intelligence & Scale - Anomaly detection overlays using TimescaleDB baseline analysis (§5.8.4). - Semantic zoom with subnet/site-level aggregation for 50k+ node deployments (§5.9). - BGP/BMP view mode with session state visualization (§5.6). - Globe/map view toggle for external traffic geolocation. --- ## 8. Open Questions | # | Question | Impact | |---|----------|--------| | 1 | Should NetFlow arc animation use `TripsLayer` (built-in) or custom luma.gl shaders? | Phase 1 — rendering approach and performance envelope | | 2 | How granular should the time-travel DVR be? Per-minute? Per-5-minutes? | Phase 3 — storage costs and query performance in TimescaleDB | | 3 | Should SRQL topology highlighting happen client-side (filter rendered data) or server-side (API returns node/edge IDs to highlight)? | Phase 3 — architecture of SRQL ↔ topology integration | | 4 | What is the aggregation strategy for semantic zoom? Subnet-based? Site/location-based? User-defined groups? | Phase 4 — UX and data modeling | | 5 | Should the MCP interface support natural-language investigation queries (e.g., "show me all traffic to Russia in the last 24 hours") that translate to SRQL + topology highlights? | Phase 4 — AI-assisted triage workflow | | 6 | Threat path rendering (§5.3.1) requires mapping NetFlow 5-tuples to physical topology hops. Is the current topology model sufficient, or do we need explicit L2 path tracing? | Phase 2 — data model completeness | --- ## 9. Competitive Differentiation Most NMS tools (LibreNMS, Zabbix, PRTG) render topology with link utilization overlays but have no security signal integration. Most SIEMs (Splunk, Elastic SIEM, Wazuh) present log events in tables and timelines but have no network topology awareness. The small number of tools that attempt both (SolarWinds, Cisco DNA Center) are proprietary, expensive, and architecturally siloed internally. ServiceRadar's approach — a single deck.gl-powered canvas that spatially correlates network state, traffic flows, and security signals — is architecturally novel in the open-source space. The combination of SRQL as a query language, MCP for AI-assisted triage, and the real-time rendering performance of deck.gl/luma.gl positions ServiceRadar to be the first open-source platform that genuinely unifies NMS and SIEM investigation workflows on a shared visual surface.

mfreeman451 added the

netflow

topology

serviceradar-web-ng

labels

2026-03-28 04:31:18 +00:00

mfreeman451 commented

2026-03-28 04:31:18 +00:00

Author

Owner

Imported GitHub comment.

Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/2912#issuecomment-3964264052
Original created: 2026-02-26T05:47:43Z

https://deck.gl/docs/api-reference/widgets/timeline-widget

Imported GitHub comment. Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/2912#issuecomment-3964264052 Original created: 2026-02-26T05:47:43Z --- https://deck.gl/docs/api-reference/widgets/timeline-widget

mfreeman451 referenced this issue

2026-04-24 02:29:40 +00:00

OpenSpec follow-up: prioritize topology and mapper fidelity #3172

mfreeman451 referenced this issue

2026-04-24 02:30:17 +00:00

OpenSpec work queue: prioritized cleanup and implementation tracker #3182