During a "let's improve SparklyPower to make it the best Survival server ever" hyperfixation phase, I've really wanted to update my server to 1.21. I waited eagerly for the shiny Paper experimental 1.21 builds, to update my server right when it is released. Who cares if it is "experimental", after all, what could possibly go wrong?

Well, that was a big mistake.

When the Paper team said that they are experimental builds that you shouldn't run in production, they mean it. The Paper experimental 1.21 builds do not have all of the optimization patches, nor they are stable. But I'm stupid and decided to ignore the experimental label.

Now I'm stuck with a server that could handle ~80 players with 20 TPS on 1.20.6, while on 1.21 it is struggling to keep 20 TPS with only 50 players online. And that makes me sad because it is hard to just shrug it off and carry on when you know that players are sad/angry because the server is lagging, especially because it was entirely my fault.

Here's the deal: Everyone says that downgrading versions is NOT supported, some even say that it is IMPOSSIBLE to downgrade and that's why you MUST backup your server before updating.

Restoring my server from a backup is unfeasible because... I don't have a backup (as I said before, I'm stupid) and even if I had it, reverting from a backup is still a pain because players will lose their progress on the server, and because our server has currency transfer between the server and Loritta, reverting from a backup is a monumental task because we also need to revert all transfers too.

And I know that they are not lying, I know Minecraft enough that I know that there are versions that you WILL end up with a corrupted world if you downgrade, like if you downgraded from 1.13 (The Flattening™) to 1.12.2. There is a real risk when downgrading. Will I get my world corrupted if I downgrade from 1.21 to 1.20.6? Who knows!

...but we have no choice but to leave our comfort zone and try to do it. If we stay on 1.21, we will need to live with angry players complaining about lag until the Paper team updates all optimization patches to 1.21, and because there's no ETA, we would be stuck with it with no end in sight.

The 1.21 to 1.20.6 Downgrading Experience

What actually happens if I copy a Paper 1.21 world to a Paper 1.20.6 server? First, I created a world with some 1.21 blocks on the world and in my inventory, a wall with old and new paintings and some pigs.

I copied the 1.21 world to a 1.20.6 server and, after starting the server, Paper refuses to load the world data because it was created with a newer version of Minecraft.

[18:14:56 INFO]: Preparing level "world"
[18:14:57 INFO]: Preparing start region for dimension minecraft:overworld
[18:14:58 WARN]: java.lang.RuntimeException: Server attempted to load chunk saved with newer version of minecraft! 3953 > 3839
[18:14:58 WARN]:        at net.minecraft.world.level.chunk.storage.ChunkSerializer.readInProgressChunkHolder(ChunkSerializer.java:150)
[18:14:58 WARN]:        at io.papermc.paper.chunk.system.scheduling.ChunkLoadTask$ChunkDataLoadTask.runOffMain(ChunkLoadTask.java:338)
[18:14:58 WARN]:        at io.papermc.paper.chunk.system.scheduling.GenericDataLoadTask$ProcessOffMainTask.run(GenericDataLoadTask.java:307)

If you dig into Paper's code, you'll find a escape hatch: The Paper.ignoreWorldDataVersion system property, which has the affectionately named JUST_CORRUPT_IT variable name.

// Paper start - Do not let the server load chunks from newer versions
private static final int CURRENT_DATA_VERSION = net.minecraft.SharedConstants.getCurrentVersion().getDataVersion().getVersion();
private static final boolean JUST_CORRUPT_IT = Boolean.getBoolean("Paper.ignoreWorldDataVersion");
// Paper end - Do not let the server load chunks from newer versions

So, what happens if you load a 1.21 world in Paper 1.20.6 with -DPaper.ignoreWorldDataVersion=true set?

Well, the results are very underwhelming! I thought that when people meant that it would corrupt the world, they meant that the server would regenerate the chunks due to invalid data or something, but nope!

However you may have noticed some oddities...

• The new painting have been reverted to the "Kebab" painting after downgrading.
• How are the Crafter and Trial Spawners still present? Weren't they added in Minecraft 1.21?
• If the Crafter and Trial Spawners are still present, then why did the new music discs vanish from my inventory?

Since Minecraft 1.19.3, Mojang likes putting the features that will be added in the next Minecraft version behind "Experiments" that you can enable by toggling the option when creating a world. So the Minecraft 1.21 blocks and items are actually already implemented in Minecraft 1.20.6, but locked behind an experiment. Because we have not enabled the experiment when creating the world, the blocks are present in the world, but they won't work. If you attempt to right click a Crafter, nothing will happen.

One of the only exceptions of blocks and items that aren't present in the "Minecraft 1.21" experiment in 1.20.6 are the three new music discs. In that case, the server pretends that these items don't exist and remove them from inventories. poof! While there weren't new blocks added in 1.21 that weren't present in the experiment, the server also removes unknown blocks from the world.

So, that's good right? That means that players that did acquire those new items (except the music discs, RIP for those players) while the server was running 1.21 won't lose their items! Well, they are paperweight now, but the player didn't lose them and they will work fine after we re-upgrade to 1.21, yay! This downgrade seems very promising!

However, world corruption may mean something different to different users. Someone may call that a world corruption because they lost data, but I would call that a "lossy downgrade".

But anyhow... what actually changed between 1.20.6 to 1.21? Maybe there is something that we haven't tested that will backfire on us after we downgrade the server, and we would get on a worse spot than we are currently in right now.

Because Minecraft 1.20.6 was released two months prior to Minecraft 1.21, there aren't actually too many data changes! Which is why we didn't experience a lot of craziness after downgrading the server. And there is a way for us to confirm this!

The "Is It Safe to Downgrade Versions?" Checklist

• Open the source server version on Minecraft Wiki and check the "Data version" number (the version you are currently running)
  • If for some reason Minecraft Wiki doesn't have it, the data version is also present on the DetectedVersion class.
• Open the target server version on Minecraft Wiki and check the "Data version" number (the version you want to downgrade to)
• Open Paper's DataConverter versions converter, each class corresponds to a data version.
• Open each class between the target version and source version.
• For each opened class, find out what exactly will break on the downgrade, then decide if it is worth to downgrade or not, or code something to migrate the new data to old data.
  • If you are a developer, you can modify the server to handle the data downgrade.

So, what changed between 1.20.6 (data version 3839) and 1.21 (data version 3953)?

• V3938 (something related to arrows?)
• V3939 (removes the minecraft:update_1_21 experiment from worlds)
• V3943 (client side changes for the background blur option)
• V3945 (entity attributes changes)

The only big one that stands out is the entity attributes changes. If you have entities in the world that do require those attributes, those attributes will be lost during the downgrade. We didn't have any entities that require those attributes anyway, so we didn't mind. Technically, if you really need to keep those attributes, you could fork Paper and rewrite the entity attributes back to 1.20.6.

Keep in mind that there isn't a surefire way of knowing if a downgrade will 100% affect you, so you need to read the code, read the snapshot changes, backup your world and test it!

Oops, Serialized ItemStacks!

And after downgrading everything may seem fine! Until one of your plugins attempt to deserializeBytes an item stack that was serializeAsBytes'd on a recent version...

[20:52:48] [Server thread/ERROR]: Error occurred while enabling DreamCore v1.0.0 (Is it up to date?)
java.lang.IllegalArgumentException: Newer version! Server downgrades are not supported!
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) ~[guava-32.1.2-jre.jar:?]
	at org.bukkit.craftbukkit.util.CraftMagicNumbers.deserializeNbtFromBytes(CraftMagicNumbers.java:579) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at org.bukkit.craftbukkit.util.CraftMagicNumbers.deserializeItem(CraftMagicNumbers.java:523) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at org.bukkit.inventory.ItemStack.deserializeBytes(ItemStack.java:773) ~[paper-mojangapi-1.20.6-R0.1-SNAPSHOT.jar:?]
	at DreamCore-shadow-dev-all.jar/net.perfectdreams.dreamcore.utils.displays.user.SparklyUserDisplayManager.load(SparklyUserDisplayManager.kt:59) ~[DreamCore-shadow-dev-all.jar:?]
	at DreamCore-shadow-dev-all.jar/net.perfectdreams.dreamcore.utils.displays.user.SparklyUserDisplayManager.start(SparklyUserDisplayManager.kt:30) ~[DreamCore-shadow-dev-all.jar:?]
	at DreamCore-shadow-dev-all.jar/net.perfectdreams.dreamcore.DreamCore.onEnable(DreamCore.kt:141) ~[DreamCore-shadow-dev-all.jar:?]
	at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:288) ~[paper-mojangapi-1.20.6-R0.1-SNAPSHOT.jar:?]
	at io.papermc.paper.plugin.manager.PaperPluginInstanceManager.enablePlugin(PaperPluginInstanceManager.java:202) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at io.papermc.paper.plugin.manager.PaperPluginManagerImpl.enablePlugin(PaperPluginManagerImpl.java:109) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:520) ~[paper-mojangapi-1.20.6-R0.1-SNAPSHOT.jar:?]
	at org.bukkit.craftbukkit.CraftServer.enablePlugin(CraftServer.java:627) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at org.bukkit.craftbukkit.CraftServer.enablePlugins(CraftServer.java:576) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at net.minecraft.server.MinecraftServer.loadWorld0(MinecraftServer.java:680) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at net.minecraft.server.MinecraftServer.loadLevel(MinecraftServer.java:442) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at net.minecraft.server.dedicated.DedicatedServer.initServer(DedicatedServer.java:340) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at net.minecraft.server.MinecraftServer.runServer(MinecraftServer.java:1141) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at net.minecraft.server.MinecraftServer.lambda$spin$0(MinecraftServer.java:328) ~[sparklypaper-1.20.6.jar:1.20.6-DEV-11380a5]
	at java.base/java.lang.Thread.run(Thread.java:1570) ~[?:?]

Uh oh! That's not good!! We do know that there weren't any item data changes between 1.20.6 and 1.21, so the items are 100% downgradeable... but this time, we don't have a escape hatch to disable the version check.

So it is time to modify Paper.

We already use our own Paper fork, SparklyPaper. I patched CraftMagicNumbers to ignore the item version check if the sparklypaper.ignoreItemDataVersion property is enabled. And yes, I kept the affectionately named JUST_CORRUPT_IT variable name.

private static final boolean JUST_CORRUPT_IT = Boolean.getBoolean("sparklypaper.ignoreItemDataVersion"); // SparklyPaper - allow item downgrades

...

@Override
public Material getMaterial(String material, int version) {
    Preconditions.checkArgument(material != null, "material == null");
    Preconditions.checkArgument(JUST_CORRUPT_IT || version <= this.getDataVersion(), "Newer version! Server downgrades are not supported!"); // SparklyPaper - allow item downgrades

...

private net.minecraft.nbt.CompoundTag deserializeNbtFromBytes(byte[] data) {
    net.minecraft.nbt.CompoundTag compound;
    try {
        compound = net.minecraft.nbt.NbtIo.readCompressed(
            new java.io.ByteArrayInputStream(data), net.minecraft.nbt.NbtAccounter.unlimitedHeap()
        );
    } catch (IOException ex) {
        throw new RuntimeException(ex);
    }
    int dataVersion = compound.getInt("DataVersion");
    Preconditions.checkArgument(JUST_CORRUPT_IT || dataVersion <= getDataVersion(), "Newer version! Server downgrades are not supported!"); // SparklyPaper - allow item downgrades
    return compound;
}

The Cherry On Top: Minecraft 1.21 features in 1.20.6

Remember when I said that 1.21 blocks and items are actually already in 1.20.6, but you can only use them if you create a world with the "Minecraft 1.21" experiment enabled? And that's true! You cannot enable the experiment on an already existing world via the Minecraft vanilla client... but we can actually enable or disable the experiment in any world by editing the world's level.dat!

To do that, copy your world's level.dat (you only need to edit your server's MAIN WORLD level.dat), open it in NBTExplorer, create a list named enabled_features and create two strings within it: One named minecraft:update_1_21 and another named minecraft:vanilla. This can also work with other Minecraft experiments, just create a world with the experiment enabled and see which features are enabled in the level.dat and replicate them on your target world.

And that's it! We didn't have any other issue after downgrading the server, and now we are back to that sweet 20 TPS.

So, next time you are thinking about updating your server to a experimental Paper build, think twice and check if all the hard hitting performance optimizations are already updated! And don't be stupid like me, make backups!

---

---