The Astro-COLIBRI alert broker streams multi-messenger transient alerts in real time: gravitational waves, gamma-ray bursts, neutrinos, fast radio bursts, optical transients and more. It is an Apache Kafka broker, so you can consume it from any language that has a Kafka client, and alerts reach you within seconds of Astro-COLIBRI processing them rather than on a polling interval.

The official Python SDK, astrocolibri, includes a consumer for this broker that handles the connection, authentication and consumer-group details for you. Using it is optional: this page documents the broker itself, so you can equally point your own Kafka client at it. The same SDK also queries the Astro-COLIBRI API, to look up the events an alert refers to and to search around them.

Both are consume only. Publishing is restricted to the Astro-COLIBRI infrastructure, so no client can inject alerts into the streams.

Getting access

Broker access is self-service: open astro-colibri.com/broker and you are taken straight to the credentials page (you are asked to sign in first if you are not already). The same page is reachable from your account page in the app under Manage broker access. You are issued a personal username and password for SASL/SCRAM authentication.

The password is shown once, at creation: the broker stores only a SCRAM verifier and cannot reveal it again. If you lose it, rotate the credential from the same page. You can also revoke it there, which takes effect immediately.

Access requires a verified account and acceptance of the terms of use. Credentials identify you personally, so treat them as any other secret and keep them out of your repositories.

Connection parameters

Everything needed to point any Kafka client at the public broker:

The connection is TLS encrypted throughout. No plaintext listener is exposed to the internet.

Available topics

The same alert is published to both the JSON and the VOEvent topic, so choose the format you want rather than subscribing to both. Every topic has 3 partitions. From Python the current list is also available at runtime as astrocolibri.TOPICS.

The VOEvent topics carry documents that validate against the IVOA VOEvent 2.0 schema. The Python SDK parses them with alert.voevent(); see its VOEvent reference.

What reaches the important stream

The important topics carry a subset of the all topics, never anything extra: every alert on important is also on all. Subscribe to important when you want the events worth reacting to without filtering the full stream yourself, and to all when you want a complete record.

Two things reach it:

• • Every newly detected event, whatever its type. The first time Astro-COLIBRI publishes an event, it is always important.
• • Updates that change the science of an event already published. An update qualifies only when it modifies at least one field that would change what you do about the event.

Updates that touch nothing in that set stay on the all topics only. That is what keeps the important stream usable: an event being revised a dozen times as bookkeeping, presentation or follow-up metadata is added does not repeatedly interrupt you.

A refined GW localisation, a neutrino gaining an association, a transient being reclassified or a burst having its redshift measured all qualify. Adding a light-curve figure or a follow-up note does not.

One further limit applies to updates: once an event is more than about a month old, its updates stay on the all topics whatever changed. Late refinements to a long-past transient are still published, they simply no longer interrupt the important stream. A newly detected event is never affected by this, even when the detection itself is historical.

Retention and delivery

Alerts are retained for 24 hours. The broker is a live stream rather than an archive, so to reach anything older use the REST API.

Your read position is tracked per consumer group on the broker, so a client that stops and restarts resumes where it left off and misses nothing published meanwhile, as long as the gap stays inside the retention window. Delivery is at least once: a client that dies between handling an alert and committing its offset will see that alert again, so make your processing idempotent if duplicates would matter to you.

Consumers sharing one group.id have the partitions divided between them, and each then receives only a share of the stream. That is what you want when scaling one pipeline across several replicas, and not what you want for two independent programs, which each need their own group.

Access scope

Your credentials are read-only and scoped by broker-side ACLs. They let you read and describe the astrocolibri.* topics, and use consumer groups whose name begins with <your-username>.

That group prefix is enforced by the broker, so a group.id without it is refused. The Python SDK adds the prefix for you; with any other client you have to include it yourself. No client is granted write access to any topic.

Using a client other than the SDK

The Python SDK is a convenience, not a requirement. Any Kafka client works with the parameters above. As a quick check from the command line, using kcat:

Note the -G alice.kcat: the consumer group carries the username prefix the ACL requires. The same holds for the Java, Go, C# and other librdkafka-based clients.