[Feedback] Need better communication around Network Upgrades

[sbvegan]

Feedback

Today is the Canyon network upgrade and some node operators missed the activation time. The result was their nodes diverging and the resolution being a resync. This is unfortunate because a resync can take a long time with our current snapshot. We do now have an archive node snapshot and a third party provided a full node snapshot.

They have noted that having an email subscription that sends out a lot of warnings for network upgrades would be helpful. Additionally they wanted to know why rollup.halt wasn't enabled by default.

Please signal below if you're interested in a network upgrade notification system.

Additional Information

No response

[arb000r]

Interested in the upgrade notification system.

Missing a notification in discord / twitter is quite easy and there are no official channels to get notified on about network upgrades on a subscription basis.

This leads to a completely bricked node and requiring a full resync from a (full) snapshot that the Optimism team do not even provide on a regular basis.

Would also like to know why rollup.halt is disabled by default?

It's kind of weird to see this sort of stuff happen on a network the size of Optimism.

[opfocus]

Information regarding snapshot data mentioned on Discord, full mode: https://discord.com/channels/667044843901681675/1080862392281481246/1195090526823776308

Achieve mode: https://discord.com/channels/667044843901681675/1080862392281481246/1195093468213694475

#44 (comment) Related Link
info: There are not plans for full node snapshots because snap sync is coming soon. That functionality should remove the need for full node snapshots. However there is a third party full node snapshot here: ethereum-optimism/docs#355 from

[arb000r]

A telegram announcements channel would be useful for these sorts of upgrade announcements as a suggestion.

[L0vd2]

I believe that the best idea is just to create a proper read only announcement channel in Discord, assign node operators with specific role and tag this role once the update info is published, so everyone always gets the alerts.

[sbvegan]

This is a great suggestion, I'll make sure the person pushing this initiative forward reads this feedback.

[moorow]

Feedback from my side:

• Notification channels regarding upgrades would be useful, TG + Email. Even if this was automated based on RSS feed of releases. (I am planning on building for myself in short run, but seems like a public service that should exist)

• rollup.halt sounds like it should be enabled by default, it is significantly better to halt a node than to brick the state when a protocol change happens

• For a chain the size of OP I would expect official monthly snapshots of full and archive nodes (Polygon have solved this problem well, https://snapshots.polygon.technology).

• Why did debug_setHead not work? Would it be possible to add something to op-geth and op-node that allows a use to safely roll back the state to a block? Maybe I am being naive but the state of a blockchain at a block is the immutable state, if you roll back it should be able to pick up from there shouldn't it?

• Are there plans to speed up sync in general? Syncing even 4 weeks of data takes a VERY long time in comparison to other L2s.

[imtipi]

should provide RSS and email subscribe
very long time ago we have a google group for node ops information announce
https://groups.google.com/a/optimism.io/g/optimism-announce
but idk why it don't use anymore

[shnoodles]

We (Airstack) would be interested in having a notification system, for sure.

Chiming in on the ideas I've seen - I am in favor of announcement only TG channel; email might require more upkeep but also good to have as we could subscribe our whole dev team as a mailing list to make sure everyone's aware about the hardforks.

We unfortunately, also missed the hardfork and had to scramble to get our OP and OP stack nodes back in sync.

[arb000r]

I agree. The amount of discord channels I'm sure everyone is a part of becomes quite exhausting. A telegram channel exists to be an important and isolated channel of communication. Email is also fine but requires a bit more overhead.

-1 Discord
+1 Email
+1 Telegram

[imtipi]

this also applies to optimism goerli deprecate,oplabs blog only say it will deprecate in Jan, but no exact date...

[sbvegan]

Hey y'all, we're going to be utilizing Discord's announcement channels to take the first steps to addressing this problem and we'll also push comms to this forum's announcement channel. We'll iterate on this and add additional channels later down the road.

cc @imtipi @mrchickenburger @shnoodles @moorow @L0vd2