Sentry is capturing error events from celery task workers but not from celery message workers

[Nitish18]

• we are using celery to handle async tasks in our django project.
• we have 3 tasks queues. All of them are defined in django base settings. Sentry is able to capture all events which are raised on triggering of any exception while executing any of async tasks.
• We are also using one ad-hoc celery message worker to process async messages as well. We publish message via using kombu and message handlers are called for each message fired. Message handlers are executed in celery message worker, but any error raised there is not captured by Sentry.
• Problem: Sentry is capturing error events from celery task workers but not from celery message workers

[sl0thentr0py]

@Nitish18 make sure sentry_sdk.init is called in each worker as soon as possible, it might not be inheriting the init from the django settings module.

[Nitish18]

Hi @sl0thentr0py , thanks for the follow up.
The celery message worker class is initialized in an independent python run-time library which we have created. We are not defining it in our django project anywhere. Our custom Library calls all callback handler function and raises an exception if there is any error in those callback functions.

Providing more info:

• how is the message worker configured, does it or kombu have any error handler? Screenshots or code snippets would be helpful.

  • we are using Celery Custom Message Consumers (https://docs.celeryq.dev/en/stable/userguide/extending.html#custom-message-consumers)

  • if we see here, we need to attach the "MyConsumerStep" class to Celery app instance: app2.steps['consumer'].add(MyConsumerStep)

  • attaching screenshot for reference on how celery worker class is configured.

  • the thing to notice is that in our django project, we are creating two Celery app instances:

    1. celery.py named "app1 = Celery()"
    2. celery_async_message_consumer.py named "app2 = Celery()"

    We are attaching this app2 with the MyConsumerStep class mentioned above.

  • also, in order to run Celery message worker we are using this command:

    • celery worker --app=.celery_async_message_consumer --broker=amqp://guest:guest@localhost:5672// --queues="async_message_queue"

• Here, we are raising "Exception" whenever there is any error while executing any message callback handler. Ref attachment 2.

• NOTE: the above worker class is defined in another python runtime library that we have created which we install via pip.
  The above worker class code snippet resides in another in-house python library and not in django project.

• the message worker is async, does the process close immediately when it is finished/has an error?

  • No. The worker stays alive just like celery task workers. It will stay alive and execute message handler callback whenever a new message arrives in the message queue.

[Nitish18]

Hi @sl0thentr0py ,
I tried your suggestion of initializing sentry_sdk.init() where the Workers are being initialized. Its working now.
thanks.

[sl0thentr0py]

glad you figured it out!