FFI/CGO sysmon-vm for osx/arm64 #2305
Closed
mfreeman451 wants to merge 6 commits from
refs/pull/2305/head into main
pull from: refs/pull/2305/head
merge into: carverauto:main
carverauto:staging
carverauto:demo/prod-release
carverauto:add-dashboard-srql-service-views
carverauto:enhance-dashboard-creator-visual-builder
carverauto:fix-release-1-2-78-ci-failures
carverauto:refactor-agent-plugin-runtime
carverauto:add-dashboard-creator
carverauto:update/plugin-system
carverauto:add-service-monitoring-foundation
carverauto:fix/cli-auth-settings-navigation
carverauto:fix/device-unknown-facet
carverauto:fix-plugin-assignment-upgrades
carverauto:fix/sweep-profile-mode-status
carverauto:fix/armis-northbound-fixes
carverauto:fix-proxmox-console-react-client-render
carverauto:fix-openbao-signing-job-token-mount
carverauto:fix-services-plugin-status-read-model
carverauto:fix-stream-status-final-chunk
carverauto:codex/fix-prod-armis-sync-icmp
carverauto:fix-demo-cnpg-operator-networkpolicy
carverauto:update/docs-cleanup
carverauto:fix-armis-northbound-raw-token-auth
carverauto:fix-armis-northbound-stuck-running
carverauto:update-manual-device-hostname-readd
carverauto:fix/observability-severity-card-links
carverauto:renovate/debian_testing_slim-testing-slim
carverauto:docs/kubernetes-ingestion-gateway
carverauto:fix/flow-collector-external-network-policy
carverauto:fix/armis-availability-and-stream-config
carverauto:refactor-fast-fresh-db-bootstrap
carverauto:fix/cnpg-not-found
carverauto:fix/docker-update/cnpg
carverauto:renovate/debian_bookworm_slim-bookworm-slim
carverauto:add-socket-firewall-ci
carverauto:add-ipv6-sweep-scanners
carverauto:fix-armis-northbound
carverauto:add-streamed-agent-config
carverauto:fix/sweep-ip-family-routing
carverauto:codex/fix-core-migration-hook-upgrade
carverauto:renovate/arc_runner
carverauto:renovate/actions_runner
carverauto:codex/remote-access-desktop-rdp
carverauto:updates/missing-fixes
carverauto:fix/batched-sweep-targets
carverauto:fix/agent-sysmon-memory-growth
carverauto:fix/armis-northbound-single-run-and-timestamps
carverauto:release/1.2.67
carverauto:fix/wasm-plugin-service-status
carverauto:fix/armis-names-string
carverauto:codex/harden-remote-access-app-tcp-followup
carverauto:codex/harden-remote-access-app-tcp
carverauto:codex/harden-gateway-proxy-auth
carverauto:codex/harden-remote-access-destroy-rbac
carverauto:codex/harden-ssh-ca-remote-access
carverauto:codex/harden-remote-access-broker-registry
carverauto:codex/harden-remote-access-approval-policy
carverauto:codex/harden-remote-access-file-transfer
carverauto:codex/harden-remote-access-attach-ticket
carverauto:fix-release-plugin-list-cleanup
carverauto:codex/harden-grpc-logger-payloads
carverauto:propose-interface-action-target-context
carverauto:add-signed-northbound-action-callbacks
carverauto:fix/reap-stale-self-scheduled-oban-jobs
carverauto:fix/device-logs-tab-async
carverauto:fix/device-metadata-summary
carverauto:fix/northbound-launch-target-auth
carverauto:fix/northbound-provider-atomic-read
carverauto:proposal/add-sample-northbound-wasm-plugin
carverauto:proposal/northbound-action-integrations
carverauto:fix/ansible-run-job-device-launch
carverauto:codex/fix-audit-events-shell-theme
carverauto:codex/fix-sweep-tcp-availability
carverauto:codex/fix-armis-names-string
carverauto:fix/update-nats-2-14
carverauto:fix/elixir-quality-release-blockers
carverauto:fix/latest-release-device-ingest-ui-bugs
carverauto:fix/release-1.2.61-ci-failures
carverauto:fix/web-ng-precommit-format
carverauto:fix/wasm-tinygo-go125
carverauto:fix/agent-file-transfer-bazel-src
carverauto:fix/release-key-stamp-root
carverauto:fix/release-1.2.59-staging
carverauto:main
carverauto:fix/agent-accept-deprecated-remote-access-config
carverauto:fix/device-results-count-facets
carverauto:fix/bazelisk-installer-retries
carverauto:fix/tinygo-host-toolchain-fetch
carverauto:add-per-agent-availability
carverauto:fix/forgejo-release-multipart-assets
carverauto:fix/agent-config-stale-session
carverauto:fix/mtr-hop-dns-resolution
carverauto:fix/hostname-only-device-create
carverauto:fix/otlp-log-metadata-sanitization
carverauto:fix/sweep-icmp-legacy-mode-classification
carverauto:add-nats-object-store-retention
carverauto:fix/helm-serviceradar-state-pvc
carverauto:harden/forgejo-ci-nonroot
carverauto:codex/expand-remote-access-teleport-parity
carverauto:fix/sweep-port-history-consistency
carverauto:fix/remote-access-ssh-feature-flag
carverauto:fix/devices-refresh-artifacts
carverauto:fix/identity-ingestion-sweep-availability
carverauto:spec/identity-cache-ingestion-correctness
carverauto:fix/sweep-mapper-promotion-stale-cache
carverauto:fix/sweep-provisional-duplicate-ip
carverauto:fix/sweep-target-invalid-ip-order
carverauto:fix/armis-large-sync-streaming
carverauto:fix/docusaurus-blog-build-date
carverauto:fix/armis-sync-compat-deep-dive
carverauto:fix/awx-controller-credential-secret
carverauto:fix/demo-release-source-branch
carverauto:demo/release-v1.2.44-source-fix
carverauto:codex/teleport-agent-routed-remote-access
carverauto:feature/agent-config-dependency-catalog
carverauto:proposal/agent-config-dependency-catalog
carverauto:bugfix/armis-credentials-save-display
carverauto:bugfix/armis-integration-credentials
carverauto:fix/web-ng-precommit-formatting
carverauto:bugfix/armis-secret-config-push
carverauto:feature/close-controllers-to-pipelines
carverauto:carverauto/extract-palisade
carverauto:feature/migrate-dashboard-cli-to-plugs
carverauto:feature/audit-history-page
carverauto:feature/migrate-controllers-to-security-pipelines
carverauto:fix/security-events-test-and-retention-worker
carverauto:feature/add-platform-security-hardening
carverauto:demo-rollout-proxmox-bazel-fix
carverauto:bug/armis-sync-issues
carverauto:add-virtualization-srql-queries
carverauto:add-proxmox-ingestion-hardening-tests
carverauto:add-ssh-private-key-credential-rules
carverauto:redact-plugin-credential-material
carverauto:add-credential-rules-settings-entry
carverauto:add-credential-rules-settings-flows
carverauto:require-network-credential-broker-grants
carverauto:preview-credential-rule-target-scope
carverauto:add-proxmox-credential-secret-preset
carverauto:harden-credential-rules-live-tests
carverauto:clarify-proxmox-plugin-credential-modes
carverauto:docs-proxmox-credential-operations
carverauto:fingerprint-proxmox-candidates
carverauto:proxmox-resource-efficiency-dashboard
carverauto:proxmox-console-security-docs
carverauto:proxmox-focused-quality-validation
carverauto:proxmox-metric-baseline-alerts
carverauto:proxmox-device-scoped-logs
carverauto:proxmox-defer-vector-log-forwarding
carverauto:proxmox-console-session-tickets
carverauto:proxmox-console-xterm-shell
carverauto:proxmox-console-websocket-broker
carverauto:proxmox-console-control-frames
carverauto:proxmox-console-agent-session-manager
carverauto:proxmox-console-plugin-pty-bridge
carverauto:proxmox-console-assignment-materializer
carverauto:proxmox-console-ssh-connector
carverauto:proxmox-console-agent-local-broker
carverauto:proxmox-console-device-actions
carverauto:proxmox-console-stream-timeouts
carverauto:proxmox-console-guest-mode-gating
carverauto:proxmox-console-ci-race-fix
carverauto:fix/falco-alert-routing-datasvc-channel
carverauto:fix-agent-release-page-bugs
carverauto:add-proxmox-device-details-summary
carverauto:add-proxmox-virtualization-ingestor
carverauto:add-virtualization-inventory-schema
carverauto:add-proxmox-infrastructure-inventory
carverauto:add-proxmox-plugin-live-smoke
carverauto:add-proxmox-local-api-smoke
carverauto:add-proxmox-credential-test-dispatch
carverauto:add-proxmox-credential-test-plan
carverauto:add-network-credential-rule-preview
carverauto:add-proxmox-plugin-inventory-details
carverauto:add-proxmox-credential-reconcile-worker
carverauto:add-proxmox-credential-assignment-materializer
carverauto:add-plugin-input-template-secret-refs
carverauto:add-proxmox-plugin-policy-inputs
carverauto:add-proxmox-wasm-plugin-scaffold
carverauto:add-network-credential-rules-model
carverauto:add-proxmox-plugin-credential-rules
carverauto:fix/rperf-rustls-provider-demo
carverauto:fix/dashboard-template-sdk-014
carverauto:fix/tinygo-go126-release
carverauto:fix/reqsign-provider-bazel-deps
carverauto:fix/release-bazel-rust-crates
carverauto:fix/core-coordinator-connection-leak
carverauto:chore/security-updates
carverauto:update/readme-versions-update
carverauto:docs/readme-dashboard-sdk
carverauto:chore/cli-0.1.5
carverauto:fix/dashboard-cli-local-map-dev
carverauto:fix/dashboard-cli-hmr-map-libraries
carverauto:chore/bump-serviceradar-cli-0.1.2
carverauto:fix/dashboard-cli-hmr-harness
carverauto:fix/helm-contour-liveview-websocket
carverauto:fix/helm-cnpg-pooler-defaults
carverauto:updates/helm-fixes
carverauto:update/fix-light-mode-analytics
carverauto:ual/dashboard-sdk-dx
carverauto:security/postgres-update
carverauto:update-falco-alert-diagnostics
carverauto:ual/react-dashboard-sdk
carverauto:fix/cnpg-saturation-fk-and-bootstrap
carverauto:ual/wifi-site-map
carverauto:fix-coraza-log-db-writer
carverauto:fix-log-viewer-syslog-processed
carverauto:plan-fieldsurvey-spatial-selection
carverauto:plan-envoy-coraza-waf
carverauto:plan-alienvault-otx-integration
carverauto:plan-fieldsurvey-sidekick-daemon
carverauto:cleanup-openspec-archive-closed-proposals
carverauto:bug-core-elx-ip-enrichment-reap
carverauto:fix-release-libcap2-pin-v125
carverauto:fix-camera-stream-gateway-route
carverauto:add-core-elx-prometheus-metrics
carverauto:add-serviceradar-observability-dashboards
carverauto:add-pgbouncer-helm-cnpg
carverauto:renovate/ghcr.io-actions-actions-runner
carverauto:feat/cluster-agent-runtime-metadata-stability
carverauto:feat/observability-shell-standardization
carverauto:feat/observability-live-log-toggle
carverauto:fix/mtr-bulk-queue-and-srql-targets
carverauto:fix/mtr-profile-protocol-keyerror
carverauto:fix/mtr-diagnostics-keyerror
carverauto:add-demo-rollout-skills
carverauto:fix-web-ng-test-support-dialyzer
carverauto:fix-web-ng-dialyzer-findings
carverauto:add-bulk-queued-mtr-diagnostics
carverauto:fix-serviceradar-core-integration-failures
carverauto:harden-agent-updater-exec-arguments
carverauto:harden-agent-release-trust-boundaries
carverauto:fix-compose-hermetic-nats-datasvc-bootstrap
carverauto:fix/openbao-release-issues
carverauto:elixir/formatting-updates
carverauto:codex/demo-cnpg-signing-release-fixes
carverauto:chore/lint-fixes
carverauto:fix/bazel-alpine-bump-and-cosign-skip
carverauto:update-event-alert-dedup-and-suppression
carverauto:armis-northbound-events
carverauto:armis-northbound-availability-updates
carverauto:push-owvypksrmooo
carverauto:codex/topology-endpoint-evidence-investigation
carverauto:codex/topology-bootstrap-and-layout-simplification
carverauto:codex/remove-ingress-nginx-edge
carverauto:fix/forgejo-ci-snmp-cache-and-ubuntu24
carverauto:bug/cnpg-mtls-failure
carverauto:bug/log-collector
carverauto:fix/forgejo-runner-labels
carverauto:fix/cargo-lock-sync
carverauto:remove-arc-runner-from-push-all
carverauto:fix-push-all-cosign-preflight
carverauto:fix-go-ci
carverauto:add-versioned-openapi-publish
carverauto:chore/forgejo-hardening
carverauto:security/k8s-hardening
carverauto:update/cluster-page-agents
carverauto:updates/helm-security-updates
carverauto:2406-feat-agent-fleet-management-secure-self-update-system
carverauto:bug/k8s-helm-deployments
carverauto:chore/k8s-arc-update
carverauto:rust-fix
carverauto:2371-analytics-stats-cards-should-abbreviate-numbers
carverauto:chore/perl-cleanup
carverauto:2942-featweb-ng-add-logs-tab-to-device-details-page
carverauto:192-feat-tftp-server
carverauto:mikemiles-dev/feature/netflow_collection
carverauto:testing
carverauto:dependabot/cargo/hostname-0.4.1
carverauto:dependabot/cargo/redis-1.0.1
carverauto:dependabot/cargo/bb8-0.9.0
carverauto:dependabot/cargo/rcgen-0.14.6
carverauto:dependabot/cargo/hyper-1.8.1
carverauto:dependabot/cargo/hyper-util-0.1.19
carverauto:dependabot/cargo/clap-4.5.51
carverauto:dependabot/cargo/thiserror-2.0.17
carverauto:dependabot/cargo/time-tz-2.0.0
carverauto:dependabot/cargo/tonic-build-0.14.2
carverauto:backup/main-pre-staging-sync-2026-04-02
carverauto:dependabot/npm_and_yarn/docs/mdast-util-to-hast-13.2.1
carverauto:815-feat-support-win32-for-agentpoller
carverauto:gh-pages
No reviewers
Labels
Clear labels
1week
2weeks
Failed compliance check
IP cameras
NATS
NATS JetStream
Possible security concern
Review effort 1/5
Review effort 2/5
Review effort 3/5
Review effort 4/5
Review effort 5/5
UI
aardvark
accessibility
amd64
api
arm64
auth
back-end
bgp
blog
bug
Something isn't working
build
checkers
ci-cd
continuous integration-continuous deployments
cleanup
cnpg
cloud-native postgres
codex
core
core service
dependencies
Pull requests that update a dependency file
device-management
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
dusk
ebpf
enhancement
New feature or request
eta 1d
eta 1hr
eta 3d
eta 3hr
feature
fieldsurvey
github_actions
Pull requests that update GitHub Actions code
go
Pull requests that update Go code
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
javascript
Pull requests that update Javascript code
k8s
log-collector
mapper
mtr
multi traceroute
needs-triage
netflow
network-sweep
observability
oracle
Oracle Linux related issues
otel
opentelemetry logs, traces, metrics
plug-in
proton
timeplus proton streaming database
python
question
Further information is requested
reddit
redhat
research
rperf
rperf-checker
rust
Pull requests that update rust code
sdk
security
serviceradar-agent
serviceradar-agent-gateway
serviceradar-web
serviceradar-web-ng
siem
snmp
sysmon
topology
ubiquiti
wasm
wontfix
This will not be worked on
zen-engine
No labels
1week
2weeks
Failed compliance check
IP cameras
NATS
Possible security concern
Review effort 1/5
Review effort 2/5
Review effort 3/5
Review effort 4/5
Review effort 5/5
UI
aardvark
accessibility
amd64
api
arm64
auth
back-end
bgp
blog
bug
build
checkers
ci-cd
cleanup
cnpg
codex
core
dependencies
device-management
documentation
duplicate
dusk
ebpf
enhancement
eta 1d
eta 1hr
eta 3d
eta 3hr
feature
fieldsurvey
github_actions
go
good first issue
help wanted
invalid
javascript
k8s
log-collector
mapper
mtr
needs-triage
netflow
network-sweep
observability
oracle
otel
plug-in
proton
python
question
reddit
redhat
research
rperf
rperf-checker
rust
sdk
security
serviceradar-agent
serviceradar-agent-gateway
serviceradar-web
serviceradar-web-ng
siem
snmp
sysmon
topology
ubiquiti
wasm
wontfix
zen-engine
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
carverauto/serviceradar!2305
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "refs/pull/2305/head"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Imported from GitHub pull request.
Original GitHub pull request: #1744
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/pull/1744
Original created: 2025-10-11T07:02:28Z
Original updated: 2025-10-12T18:22:12Z
Original head: carverauto/serviceradar:1743-chore-investigate-using-fficgo-instead-of-exec-for-hostfreq-mac
Original base: main
PR Type
Enhancement
Description
Replace exec-based hostfreq with embedded CGO/FFI implementation
Integrate Objective-C++ IOReport sampler directly into Go binary
Remove standalone hostfreq binary and launchd service dependencies
Simplify macOS deployment to single checker process
Diagram Walkthrough
File Walkthrough
6 files
Replace exec-based hostfreq with embedded collectorAdd CGO bridge for macOS IOReportAdd stub implementation for non-Darwin platformsConvert standalone binary to CGO libraryAdd C header for CGO interfaceAdd C++ wrapper for CGO compilation1 files
Update test mocks for embedded collector10 files
Remove hostfreq binary installation and serviceRemove hostfreq from packaging pipelineUpdate CI paths for moved C++ codeRemove hostfreq build targetsRemove hostfreq Bazel targetsRemove hostfreq MakefileRemove hostfreq launchd service definitionRemove hostfreq environment variableRemove hostfreq from Bazel packagingAdd CGO and C++ library targets4 files
Update documentation for embedded samplerUpdate documentation for embedded approachUpdate architecture documentationUpdate runbook for simplified deploymentImported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/1744#issuecomment-3393003618
Original created: 2025-10-11T07:03:18Z
PR Compliance Guide 🔍
Below is a summary of compliance checks for this PR:
FFI memory management
Description: C-allocated buffers are returned to Go via pointers (
hostfreq_collect_json) and requireprecise lifetime management; while
hostfreq_freeis provided and used in Go, any futuremisuse or missing frees would cause memory leaks or use-after-free—careful auditing of all
call paths that receive
out_json/out_erroris required.hostfreq_darwin.mm [421-486]
Referred Code
Privilege requirements
Description: Use of private macOS IOReport APIs requires elevated privileges; misconfiguration could
lead to permission errors or encourage users to run the service with excessive privileges,
increasing attack surface—documented, but operational hardening is advised.
hostfreq_darwin.mm [338-366]
Referred Code
🎫 #1743
returns expected frequencies.
with network/firewall constraints.
Codebase context is not defined
Follow the guide to enable codebase context checks.
No custom compliance provided
Follow the guide to enable custom compliance check.
Compliance status legend
🟢 - Fully Compliant🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/1744#issuecomment-3393004225
Original created: 2025-10-11T07:04:18Z
PR Code Suggestions ✨
Explore these optional code suggestions:
Provide an error on memory allocation failure
In
hostfreq_collect_json, ifmallocfails, set an "out of memory" message inout_errorto provide a more specific error to the Go caller.pkg/cpufreq/hostfreq_darwin.mm [437-443]
Suggestion importance[1-10]: 6
__
Why: The suggestion correctly identifies a gap in error reporting where a memory allocation failure is not explicitly communicated to the caller, and the proposed fix improves diagnostics for the Go-side consumer.
Remove redundant nil check for C string
Remove the redundant
nilcheck foroutJSONincollectViaHostfreq, asC.GoString(nil)already produces an empty string that is handled by a subsequentcheck.
pkg/cpufreq/hostfreq_darwin.go [79-86]
Suggestion importance[1-10]: 2
__
Why: While the suggestion is technically correct that
C.GoString(nil)returns an empty string, removing thenilcheck would merge two distinct error conditions into one, losing the specific error message "hostfreq returned no data".Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/1744#issuecomment-3393684923
Original created: 2025-10-11T21:59:10Z
CI Feedback 🧐
(Feedback updated until commit
github.com/carverauto/serviceradar@d3813e348d)A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
Action: build
Failed stage: Test [❌]
Failed test name: ""
Failure summary:
The Bazel build failed due to environment/toolchain incompatibilities on the runner:
- GCC is too
old for multiple dependencies (Abseil, Protobuf, etc.). Many C++ compilations error with "This
package requires GCC 7 or higher." and Protobuf asserts "GCC 7.3 and newer" (e.g.,
external/abseil-cpp+/absl/base/policy_checks.h:57:2,external/protobuf+/src/google/protobuf/compiler/BUILD.bazel:115).- The C toolchain resolution also
had issues (initial error: unable to find a CC toolchain for platform
@@rules_go+//go/toolchain:darwin_arm64, exec platform@@//build/rbe:rbe_platformatexternal/rules_cc+/cc/BUILD:145:19), indicating misconfigured/unsupported toolchain selection inremote execution.
- Multiple C compilations failed due to "absolute path inclusion(s) found" (e.g.,
external/zlib+/BUILD.bazel:91:11,third_party/cwalk/BUILD.bazel:7:11,protobuf+/third_party/utf8_range), where headers from/usr/lib/gcc/x86_64-linux-gnu/5/...wereincluded via absolute paths, which Bazel forbids under the configured toolchain.
- An OCaml PPX step
failed due to missing GLIBC version on the runner:
/lib/x86_64-linux-gnu/libm.so.6: version'GLIBC_2.29' not foundwhen runningexternal/tools_opam++opam+opam.lwt_ppx/lib/ppx.exe(BUILD.bazel:150:13), indicating the host system glibc is too old for the provided prebuilt PPX.
Overall, the CI runner environment (GCC 5.x, older glibc) and toolchain configuration are
incompatible with the repository’s dependencies and Bazel toolchain expectations, causing widespread
compile-time failures.
Relevant error logs:
Imported GitHub PR comment.
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/pull/1744#issuecomment-3395195960
Original created: 2025-10-12T18:22:09Z
abandoning, pulled these changes into a different branch, fixed issues and merged, this is stale.
Pull request closed