Tag: GitHub

  • GitHub Weekly: Memory Architecture, Health Probes, and the Quiet Work of Production Hardening

    GitHub Weekly: Memory Architecture, Health Probes, and the Quiet Work of Production Hardening

    The week of June 15-22 was not defined by a single dramatic event. Instead, it was the kind of week that separates platforms that merely work from platforms that hold up under sustained pressure. Across 50 repositories, over 100 events — commits, pull requests, issues — pushed forward several long-running threads: memory architecture, health monitoring, security governance, and the unglamorous but essential work of keeping production infrastructure honest.

    What Happened

    hermes-mgmt — Hardening the Core

    The hermes-mgmt repository remained the busiest node in the network, with 20-plus commits, 20 PRs, and 13 issues. The dominant theme was memory system reliability. A significant fix (PR covering issues #573, #574, #575) hardened dual_memory.py across three dimensions: Ollama-first Mem0 configuration, deterministic Letta archival behaviour, and Qdrant vector dimension alignment. These are the kinds of fixes that don’t make headlines but prevent the subtle data corruption that erodes trust in AI systems over time.

    A related fix (issue #411) corrected a memory drift check that was comparing the wrong Letta data — core-memory passages instead of ARCHIVAL passages. This is a telling detail: as memory architectures grow more layered (core, archival, vector), the surface area for misaligned reads increases. Catching this before it caused silent data degradation matters.

    Health probes got a major upgrade with PR #568, which introduced functional health checks for services that report as “green” but are actually broken — Letta, n8n, Langfuse, Qdrant, Gateway, and Ollama. This is a pattern anyone running distributed systems will recognise: the dashboard shows all green, but something is quietly failing. Functional probes go beyond “is the process running?” to “does the service actually respond correctly?” — a distinction that matters enormously in production.

    Secrets management continued to mature. PR #552 delivered a comprehensive secrets management architecture built around Bitwarden and HashiCorp Vault, while PR #527 removed committed default secrets from the Langfuse compose configuration. These are the foundational security practices that need to be in place before scale makes them painful to retrofit.

    On the cron and scheduling side, the system now runs 38 cron jobs with 36 healthy — a 95% health rate that reflects the cumulative effect of weeks of hardening work. PR #504 corrected cron exit-code semantics so that findings and alerts are no longer misinterpreted as failures, which was causing unnecessary noise in the monitoring pipeline.

    Several open issues point to the next layer of work: Telegram flood-control protection (#583), gateway ungraceful shutdown (#582), Ollama-agent sequential dispatch performance (#580), and a Langfuse Cost Report cron that’s been disabled due to stale API keys (#576). None of these are emergencies, but they represent the known gaps that get addressed in order of priority.

    hamnet — Infrastructure Truth

    The hamnet repository, which manages infrastructure automation, saw 9 commits and 10 PRs with a focus on hosting hardening and monitoring.

    SSH hardening was codified for the VPS fleet (PR #131), including a dynamic-IP allowlist failsafe — a practical safeguard for environments where IP addresses shift. Vhost routing was hardened after www.theitrevolution.co.uk was found serving the wrong site, a class of misconfiguration that can be difficult to spot without deliberate verification.

    On the monitoring side, an Alertmanager webhook receiver was added with X-Webhook-Token authentication (PR #87), and a new Mac AI Server dashboard was built for the Ollama era (PR #134). The dashboard work reflects a broader shift: as the AI infrastructure stack evolves (Ollama replacing previous model servers), the monitoring layer needs to evolve in parallel or it becomes a liability rather than an asset.

    An open issue (#95) flags an SSL certificate SAN mismatch for www.richardham.co.uk — the kind of thing that works until it doesn’t, usually at the worst possible moment.

    hermes-agent — Gateway Stability

    The hermes-agent repository received 10 commits focused on gateway reliability. The command-line matcher was hardened, Windows restart no longer causes a silent outage, and the gateway now refreshes its cached agent max_iterations from the current config rather than stale values. A fix for dict choice unwrapping in the clarify function rounds out a set of small but meaningful stability improvements.

    These are the fixes that users never notice — because they prevent the failures that would have been noticed. Silent outages and stale config caches are particularly insidious because they can persist for extended periods before manifesting as user-visible problems.

    hermes-securescore — Evidence Collection

    The SecureScore project advanced with 9 commits and 2 PRs, adding both a Docker collector and a Hermes config collector for security evidence. A high-risk action approval playbook was also added, formalising the governance process for sensitive operations. The project recorded its live activation this week — a milestone that moves it from development into operational use.

    hermes-voice-satellite — Laying Groundwork

    The voice satellite project saw 9 commits establishing the MVP build plan, an operations runbook, a Hermes voice satellite API contract, and a Termux bootstrap script for Android (S24). A native Android app placeholder was also added. This is early-stage infrastructure work — the kind of foundation that needs to exist before feature development can accelerate.

    project-atlas-foundation — Launch Readiness

    Project Atlas received 10 commits focused on governance and launch preparation: a PR template, MIT licence, security policy, CODEOWNERS, a safety checklist, research workflow, promotion process, CI configuration, and hardened lifecycle guides. Scoring of proposals #3-#7 was completed alongside shaping docs and a launch checklist update. This is the organisational scaffolding that turns a codebase into a project other people can contribute to.

    AgentRadar — New Arrival

    AgentRadar received its initial commit this week. Details are sparse, but a new repository appearing in the portfolio is always worth noting — it represents a new thread that will either find its place or be retired honestly.

    richardham-web-and-Brand — Content Pipeline

    The web-and-brand repository saw 10 commits and 5 PRs, primarily focused on blog publishing and content pipeline maintenance. Four new blog posts in the agentic AI series were added with proper date spacing, and a build fix declared window.__calComLoaded to resolve a strict type-check failure. A WordPress publishing blocker was documented (VPS SSH key issue), which is the kind of honest infrastructure transparency that keeps content pipelines reliable.

    Key Takeaways

    Memory architecture is the new frontier. The volume of work on dual-memory hardening, Letta archival alignment, and Qdrant dimension matching signals that the memory subsystem has become a first-class concern. As AI agents handle longer and more complex tasks, the reliability of their memory layer directly determines the reliability of everything built on top of it. The drift-check fix — comparing the right data — is a small change with outsized implications.

    Functional health probes close a critical gap. The distinction between “process is running” and “service is working” is one of the most common blind spots in monitoring. Adding functional probes for six core services moves the observability stack from surface-level to meaningful. This is the kind of investment that pays for itself the first time it catches a green-but-broken service before a user does.

    Security governance is becoming systematic, not reactive. Between the secrets management architecture, the SecureScore evidence collectors, the high-risk action approval playbook, and the removal of committed defaults, the pattern is clear: security is being built into the development process rather than bolted on after incidents. This is the maturation path every platform needs to follow.

    Infrastructure truth matters. The hamnet work — SSH hardening, vhost routing fixes, SSL certificate monitoring, dashboard reconciliation — is the unglamorous foundation that everything else depends on. When www.theitrevolution.co.uk serves the wrong site, no amount of AI sophistication compensates. Keeping the infrastructure layer honest is a continuous discipline, not a one-time project.

    The content pipeline is converging with the platform. The parallel work on blog publishing, brand positioning, and the agentic AI content series is not separate from the technical work — it is how the technical work becomes visible and valuable. A hardened platform with strong governance needs an equally strong narrative around it.

    Looking Ahead

    The open issues across the portfolio paint a clear picture of next week’s priorities: resolve the Telegram flood-control design, address the gateway ungraceful shutdown path, fix the Langfuse Cost Report cron’s stale API keys, and close the SSL certificate SAN mismatch. On the infrastructure side, the Mac AI Server Ollama dashboard and metrics pipeline need to land, and the remaining open PRs in the web-and-brand repository need to progress through review.

    The velocity of the last week is notable not for its drama but for its consistency. Every repository moved forward. No single event dominated. That is what a healthy development portfolio looks like in practice — not a single sprint, but sustained, disciplined progress across every layer of the stack.

  • GitHub Weekly: Memory Architecture, Health Probes, and the Quiet Work of Production Hardening

    GitHub Weekly: Memory Architecture, Health Probes, and the Quiet Work of Production Hardening

    The week of June 15-22 was not defined by a single dramatic event. Instead, it was the kind of week that separates platforms that merely work from platforms that hold up under sustained pressure. Across 50 repositories, over 100 events — commits, pull requests, issues — pushed forward several long-running threads: memory architecture, health monitoring, security governance, and the unglamorous but essential work of keeping production infrastructure honest.

    What Happened

    hermes-mgmt — Hardening the Core

    The hermes-mgmt repository remained the busiest node in the network, with 20-plus commits, 20 PRs, and 13 issues. The dominant theme was memory system reliability. A significant fix (PR covering issues #573, #574, #575) hardened dual_memory.py across three dimensions: Ollama-first Mem0 configuration, deterministic Letta archival behaviour, and Qdrant vector dimension alignment. These are the kinds of fixes that don’t make headlines but prevent the subtle data corruption that erodes trust in AI systems over time.

    A related fix (issue #411) corrected a memory drift check that was comparing the wrong Letta data — core-memory passages instead of ARCHIVAL passages. This is a telling detail: as memory architectures grow more layered (core, archival, vector), the surface area for misaligned reads increases. Catching this before it caused silent data degradation matters.

    Health probes got a major upgrade with PR #568, which introduced functional health checks for services that report as “green” but are actually broken — Letta, n8n, Langfuse, Qdrant, Gateway, and Ollama. This is a pattern anyone running distributed systems will recognise: the dashboard shows all green, but something is quietly failing. Functional probes go beyond “is the process running?” to “does the service actually respond correctly?” — a distinction that matters enormously in production.

    Secrets management continued to mature. PR #552 delivered a comprehensive secrets management architecture built around Bitwarden and HashiCorp Vault, while PR #527 removed committed default secrets from the Langfuse compose configuration. These are the foundational security practices that need to be in place before scale makes them painful to retrofit.

    On the cron and scheduling side, the system now runs 38 cron jobs with 36 healthy — a 95% health rate that reflects the cumulative effect of weeks of hardening work. PR #504 corrected cron exit-code semantics so that findings and alerts are no longer misinterpreted as failures, which was causing unnecessary noise in the monitoring pipeline.

    Several open issues point to the next layer of work: Telegram flood-control protection (#583), gateway ungraceful shutdown (#582), Ollama-agent sequential dispatch performance (#580), and a Langfuse Cost Report cron that’s been disabled due to stale API keys (#576). None of these are emergencies, but they represent the known gaps that get addressed in order of priority.

    hamnet — Infrastructure Truth

    The hamnet repository, which manages infrastructure automation, saw 9 commits and 10 PRs with a focus on hosting hardening and monitoring.

    SSH hardening was codified for the VPS fleet (PR #131), including a dynamic-IP allowlist failsafe — a practical safeguard for environments where IP addresses shift. Vhost routing was hardened after www.theitrevolution.co.uk was found serving the wrong site, a class of misconfiguration that can be difficult to spot without deliberate verification.

    On the monitoring side, an Alertmanager webhook receiver was added with X-Webhook-Token authentication (PR #87), and a new Mac AI Server dashboard was built for the Ollama era (PR #134). The dashboard work reflects a broader shift: as the AI infrastructure stack evolves (Ollama replacing previous model servers), the monitoring layer needs to evolve in parallel or it becomes a liability rather than an asset.

    An open issue (#95) flags an SSL certificate SAN mismatch for www.richardham.co.uk — the kind of thing that works until it doesn’t, usually at the worst possible moment.

    hermes-agent — Gateway Stability

    The hermes-agent repository received 10 commits focused on gateway reliability. The command-line matcher was hardened, Windows restart no longer causes a silent outage, and the gateway now refreshes its cached agent max_iterations from the current config rather than stale values. A fix for dict choice unwrapping in the clarify function rounds out a set of small but meaningful stability improvements.

    These are the fixes that users never notice — because they prevent the failures that would have been noticed. Silent outages and stale config caches are particularly insidious because they can persist for extended periods before manifesting as user-visible problems.

    hermes-securescore — Evidence Collection

    The SecureScore project advanced with 9 commits and 2 PRs, adding both a Docker collector and a Hermes config collector for security evidence. A high-risk action approval playbook was also added, formalising the governance process for sensitive operations. The project recorded its live activation this week — a milestone that moves it from development into operational use.

    hermes-voice-satellite — Laying Groundwork

    The voice satellite project saw 9 commits establishing the MVP build plan, an operations runbook, a Hermes voice satellite API contract, and a Termux bootstrap script for Android (S24). A native Android app placeholder was also added. This is early-stage infrastructure work — the kind of foundation that needs to exist before feature development can accelerate.

    project-atlas-foundation — Launch Readiness

    Project Atlas received 10 commits focused on governance and launch preparation: a PR template, MIT licence, security policy, CODEOWNERS, a safety checklist, research workflow, promotion process, CI configuration, and hardened lifecycle guides. Scoring of proposals #3-#7 was completed alongside shaping docs and a launch checklist update. This is the organisational scaffolding that turns a codebase into a project other people can contribute to.

    AgentRadar — New Arrival

    AgentRadar received its initial commit this week. Details are sparse, but a new repository appearing in the portfolio is always worth noting — it represents a new thread that will either find its place or be retired honestly.

    richardham-web-and-Brand — Content Pipeline

    The web-and-brand repository saw 10 commits and 5 PRs, primarily focused on blog publishing and content pipeline maintenance. Four new blog posts in the agentic AI series were added with proper date spacing, and a build fix declared window.__calComLoaded to resolve a strict type-check failure. A WordPress publishing blocker was documented (VPS SSH key issue), which is the kind of honest infrastructure transparency that keeps content pipelines reliable.

    Key Takeaways

    Memory architecture is the new frontier. The volume of work on dual-memory hardening, Letta archival alignment, and Qdrant dimension matching signals that the memory subsystem has become a first-class concern. As AI agents handle longer and more complex tasks, the reliability of their memory layer directly determines the reliability of everything built on top of it. The drift-check fix — comparing the right data — is a small change with outsized implications.

    Functional health probes close a critical gap. The distinction between “process is running” and “service is working” is one of the most common blind spots in monitoring. Adding functional probes for six core services moves the observability stack from surface-level to meaningful. This is the kind of investment that pays for itself the first time it catches a green-but-broken service before a user does.

    Security governance is becoming systematic, not reactive. Between the secrets management architecture, the SecureScore evidence collectors, the high-risk action approval playbook, and the removal of committed defaults, the pattern is clear: security is being built into the development process rather than bolted on after incidents. This is the maturation path every platform needs to follow.

    Infrastructure truth matters. The hamnet work — SSH hardening, vhost routing fixes, SSL certificate monitoring, dashboard reconciliation — is the unglamorous foundation that everything else depends on. When www.theitrevolution.co.uk serves the wrong site, no amount of AI sophistication compensates. Keeping the infrastructure layer honest is a continuous discipline, not a one-time project.

    The content pipeline is converging with the platform. The parallel work on blog publishing, brand positioning, and the agentic AI content series is not separate from the technical work — it is how the technical work becomes visible and valuable. A hardened platform with strong governance needs an equally strong narrative around it.

    Looking Ahead

    The open issues across the portfolio paint a clear picture of next week’s priorities: resolve the Telegram flood-control design, address the gateway ungraceful shutdown path, fix the Langfuse Cost Report cron’s stale API keys, and close the SSL certificate SAN mismatch. On the infrastructure side, the Mac AI Server Ollama dashboard and metrics pipeline need to land, and the remaining open PRs in the web-and-brand repository need to progress through review.

    The velocity of the last week is notable not for its drama but for its consistency. Every repository moved forward. No single event dominated. That is what a healthy development portfolio looks like in practice — not a single sprint, but sustained, disciplined progress across every layer of the stack.

  • HeliOS-Studio: AI Startup Studio Ignites

    t

    After a few years of building AI tooling, I hit a point where the stack stopped feeling like scaffolding. It started to look like a product in its own right.

    That is a strange moment. You begin by solving a narrow operational problem, then realise the workflow you built to support the work is now valuable enough to stand on its own.

    The shape of it

    The setup was simple in principle:

    • one layer for orchestration
    • one layer for safe execution
    • one layer for inference
    • one layer for content and delivery

    The names changed over time. The pattern did not.

    What the studio produced

    The useful output was not a single breakthrough. It was a steady stream of small, shippable things: business plans, MVP outlines, content drafts, docs, and working repos.

    That changed how I thought about progress. Instead of asking, “Can the system automate this?” I started asking, “Can the system help turn this into something a person could actually use?”

    The takeaway

    Infrastructure is only boring until it starts making decisions for you.

    When the workflow is good enough, the tooling stops being background noise. It becomes part of the offer.

  • How I Stopped AI Workflows From Sprawling

    g

    By mid-2024 I was juggling several AI tools at once. The work was useful, but the coordination started to eat the time I was trying to save. Every extra prompt, every follow-up, every half-finished idea added friction.

    So I changed the shape of the work. Ideas became GitHub issues. Research happened first. Code changes went through review. Nothing moved forward without a human looking at it.

    The workflow

    Idea -> GitHub issue -> research -> code -> review -> deploy
    

    The important part was not the diagram. It was the discipline around it.

    • Every item carried a priority and a budget.
    • The review step stayed human.
    • The system ran overnight so I could review the output in the morning instead of reacting in real time.

    What it changed

    The main win was not speed for its own sake. It was clarity. I spent less time babysitting the process and more time making decisions that mattered.

    A rough summary of the difference:

    Metric Before After
    Code hours per week 20h 4h
    Monthly token spend £120 £12
    Projects shipped per month 1 4
    GitHub commits per month 45 200+

    What made it work

    1. Keep the routing local when you can.
    2. Keep write access scoped.
    3. Treat the repository as the source of truth.

    That combination is what made the setup durable. Not the tools, really. The rules.

    I still think that is the part people miss. AI does not remove the need for process. It makes process more obvious.

  • GitHub AI Boom: 65k+ Projects Spark My Homelab

    l

    In late 2023 the pace of AI tooling changed fast. The cloud options were exciting, but they also made the cost and privacy trade-offs impossible to ignore.

    So I started moving experiments into a local homelab.

    The setup

    The early setup was basic:

    • a GPU-backed VM
    • local model storage
    • simple monitoring from day one

    That was enough to prove the idea.

    What I learned

    Local inference was not perfect, but it was good enough for a surprising amount of work. The bigger models stayed useful for harder reasoning tasks, but a local model could handle a lot of the boring stuff cheaply.

    That changed how I thought about the system. The cloud stopped being the default. It became the exception.

    The real lesson

    Once you run a few workflows locally, the shape of the problem changes. Privacy becomes easier to reason about. Costs become predictable. And you start writing down the routing rules because the routing rules themselves become valuable.

    That was the start of a much longer shift for me, but the important bit was simple: the local stack made the work feel more deliberate.