Skip to content

fix: handle removing closed connections#254

Merged
harryrackmil merged 1 commit intomainfrom
harry/handle_removing_closed_connection
Mar 2, 2026
Merged

fix: handle removing closed connections#254
harryrackmil merged 1 commit intomainfrom
harry/handle_removing_closed_connection

Conversation

@harryrackmil
Copy link
Contributor

@harryrackmil harryrackmil commented Feb 27, 2026

Change

When a broker connection is removed for a publisher agent, it may be disconnect on stork's side before the publisher agent attempts to remove the connection. We should handle this case and not call .Remove() if the connection is already closed, since it will cause a nil panic.

Testing

  • unit tests
  • test in dev
  • QA going from 1 -> 0 connections
  • QA going from 2 -> 1 connections

if !exists {
r.outgoingConnectionsLock.RLock()
outgoingConnection := r.outgoingConnectionsByBroker[url]
outgoingConnection, outgoingConnectionExists := r.outgoingConnectionsByBroker[url]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the connection is closed on the stork side, the onClose() will get called which removes this connection from this map. So we need to handle the possibility that the connection has already been removed from the map and not call .Remove() in that case

brokerPublishUrl1: {"BTCUSD": {}},
}, nil).Once()
runner.UpdateBrokerConnections()
time.Sleep(100 * time.Millisecond)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first create the connections by mocking a registry response and calling update connections

time.Sleep(100 * time.Millisecond)

// close connection (disconnected by stork)
runner.outgoingConnectionsByBroker[brokerPublishUrl1].onClose()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call the onClose for this connection (which removes it from the relevant map)

runner.outgoingConnectionsByBroker[brokerPublishUrl1].onClose()

// removing the only connection means that we get an auth error when hitting the broker
registry.On("GetBrokersForPublisher", mock.Anything).Return(nil, errors.New("fake error")).Once()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in practice removing the last connection means this returns an error, so model that behavior in test

// removing the only connection means that we get an auth error when hitting the broker
registry.On("GetBrokersForPublisher", mock.Anything).Return(nil, errors.New("fake error")).Once()

runner.UpdateBrokerConnections()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this call was panicking before I updated runner.go

@harryrackmil harryrackmil marked this pull request as ready for review March 2, 2026 17:16
@harryrackmil harryrackmil requested a review from akawalsky March 2, 2026 17:27
@harryrackmil harryrackmil merged commit dfb19d9 into main Mar 2, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants