Firmware v1.11.0 released

Firmware v1.11.0 released

This release has been long overdue. In the past years things had become quite chaotic for me, and I have been able to achieve much less then I had anticipated to. In doing so I have not only let all of you down but KRIKzz as well. For that, I would like to offer my sincere apologies.

This release does not contain any new features with respect to v1.11.0b1 but some bug fixes and support for a new hardware platform (FXPAK Pro Rev. D).

FXPAK Pro Rev. D is a hardware refresh replacing the main controller due to supply chain issues. The new controller is roughly as powerful as the old one, give or take a few percent. While the CPU is basically the same, all the peripheral devices need to be programmed differently, and that is what makes up the bulk of the changes in this release while having no positive impact on the user other than maintaining the status quo. Ah well πŸ™‚

All other parts, including the FPGA, remain the same.

Download here: https://sd2snes.de/files/sd2snes_firmware_v1.11.0.zip

Since the previous release was a beta, the following list of changes is compiled with respect to v1.10.3, not v1.11.0b1 so the rest of this post will be largely identical to the v1.11.0b1 one. New changes from v1.11.0b1 to v1.11.0 are listed at the top respectively.

The tribute screen for Near is still contained in this release, even if it has been a long time now. I wanted it to be at least in one official release.

Features added:

  • First firmware release to support FXPAK Pro Rev. D
  • [All] USB (usb2snes) support by RedGuy
  • [All] save state support (via USB/savestate2snes) by acmlm, total, Myria, RedGuy, FURiOUS, ikari_01
  • [All] stand-alone save state features by FURiOUS
  • [All] some compatibility improvements and optimizations for stand-alone save states by ikari_01
    • Add support for YAML list items for multiple savestate_fixes entries per game
    • Add support for simple bitwise operations on savestate_fixes patches
    • Add support for verbatim code execution on savestate_fixes patches – notably fixes Star Ocean (decompressed) save states (however the save state hook still needs CPU time optimization to prevent music glitches in Star Ocean.)
    • Add IRQ support on top of NMI for the save state hook – this enables save states on Out Of This World and possibly other games that only use IRQ. Also fixes controller input capture on a number of games so the game-specific input hacks could be eliminated.
    • Moved save state code outside of the USB hook area to make room for USB hook execution.
  • [All] SGB support by Redacted173.
  • [All] favorite games list by freelancer42

Fixes:

  • [All] Fixed savestates crashing occasionally when loading (and sometimes saving)
  • [All] Fixed Star Fox locking up during sortie sequence when in-game hooks are enabled
  • [All] Updated FURiOUS’s Savestates Readme to reflect changes pertaining to In-game hook
  • [All] Fixed data caching bug in MSU1 data extension which could cause wrong data to be streamed
  • [All] Savestates: do not try to capture data that is already in cartridge space anyway (e.g. SRAM). Fixes text distortion in Near’s Bahamut Lagoon translation when loading a state.
  • [All] Fixed YAML config parser to support comment signs after list start items
  • [All] Fixed SNES open-bus contamination (fixes sprite flickering in Near’s Bahamut Lagoon translation, certain speed runs that rely on open bus behaviour)
  • [All] (hopefully) fixed rapid data line toggling sometimes resulting in address line glitches, causing random crashes in extreme cases, and occasional garbled graphics on SMRPG among others
  • [All] Fixed data integrity problems when loading BSX games

Known issues:

  • Due to integration of the savestate handler into the in-game hook, the SNES CPU cycle cost for the in-game hook has increased. This reintroduces some issues when in-game hook+savestates are enabled, such as a corrupted Equip menu in Final Fantasy III/VI. I am planning to do an overhaul of the savestate code which will address this issue as well.
  • When in-game savestates are enabled and set to the default button combination, pushing the buttons to temporarily or permanently disable the in-game hook will result in a crash. This happens due to overlap with the savestate button combination.
  • When in-game savestates are enabled and the “load state” button combination is pressed, the handler will try to load a state even if no state exists yet for the current game. This will lead to a crash and sometimes results in a still image being displayed from a previously played game’s savestate.

Usage Notes:

SGB

Note: SGB support requires at least two additional supplementary files:

  • sgb#_boot.bin: The internal boot ROM of the SGB-CPU
  • sgb#_snes.bin: the SGB BIOS ROM that runs on the SNES when you use the SGB.
  • ‘#’ in the filename is replaced with a 1 or 2 depending on the BIOS version you select in the SGB Settings menu, e.g. if you select BIOS version 1 it will look for sgb1_boot.bin / sgb1_snes.bin.
  • The status of the required files can be viewed on the System Information screen.
  • SGB In-game hooks and save states only work when the files match known samples.
  • Please also refer to README.SGB.txt in the sd2snes folder.

Save States

Important notes:

  1. Save states are currently only available on games that do not use expansion chips.
  2. Save states are not currently expected to be fully compatible with 100% of games. If you find a game that does not work with save states, please report – I will be working to improve compatibility.
  3. Save states do not include the state of the SNES’s audio subsystem (APU) due to technical restrictions of the SNES hardware design. Therefore music, sound effects, etc. will remain unchanged after loading a state. Some games maintain close synchronisation between game logic and APU state, requiring (usually) a 1-byte patch to bring them up to date with the APU. These patches are maintained in the file savestate_fixes.yml. For anyone proficient in debugging SNES games who wants to add their own patches, possible formats for patches are outlined in the file.
  4. If you succeed in making a save state fix patch for a new game, please share it πŸ™‚
  5. To use standalone (non-USB) in-game save states, “In-game hook” must be enabled in the “In-game Settings” menu, and “In-game savestates” must be enabled in the “Savestates Settings” menu.
  6. Please also refer to README.Savestates.FURiOUS.md in the sd2snes folder.

When the “Savestate slots” option is enabled, one of four save state slots can be selected in-game using a button combination.

“Load delay (frames)” is used to set a delay after loading a save state before returning control to the game. This gives you some time to let go of the save state buttons and get your fingers back into the normal playing position.

Save states can be saved and loaded in-game using button combinations on controller 1:

ButtonsFunction
Start+LLoad state
Start+RSave state
Select+D-PadSelect save state slot

NOTE: In-game Savestates button combinations can be overridden per game. Please check savestate_inputs.yml for any entries if the usual buttons aren’t working for you on a specific game.

Using the D-pad one of four save state slots may be selected (Default: Slot 1):

ButtonsSlot number
Select+UpSlot 1
Select+RightSlot 2
Select+DownSlot 3
Select+LeftSlot 4

Favorites List

The favorites list can be viewed from the Main Menu (X button).

To add a game to the favorites list, simply navigate to it in the file browser and press Y to pull up a context menu that displays actions that can be performed on the current selection. (Right now this menu only contains a single entry – “Add to favorites”.

To remove a game from the favorites list, open the “Favorite games” list from the Main Menu, select the entry to remove, and again use the context menu (Y button) and select “Remove from favorites”.

Reporting issues

Please report any issues preferrably on Github, or in the comment section, on Discord (#fxpak-general), or Twitter (@orzvektor).

ATTN: Anyone who received their FXPAK Pro just now

ATTN: Anyone who received their FXPAK Pro just now

It was brought to my attention that the newly shipped FXPAK Pro carts only come up with a black screen. This is due to the currently released firmware being incompatible with the new hardware. This is simply bad timing and an oversight on my part.

I was crunching to make USB work with the new Rev.D hardware in the past three weeks – at the same time failing to realize that the cartridges don’t ship with cards containing the firmware build I sent to krikzz for burn-in testing earlier. The new hardware, besides using a new microcontroller, also needs slightly adjusted FPGA core files due to differences in clocking so the existing ones are not compatible.

I was going to release the 1.11.0 firmware right about now anyway but I’m trying to tackle a savestates issue first, where it sometimes crashes after saving or loading a state in standalone savestate mode.
QUSB2SNES / SaveState2Snes appears to work fine though.
Here is the current build. I will publish a proper release within the next few days.

Note: This is for Rev.D owners only who have been currently left with a non-operating cartridge. Final release with support for all sd2snes/FXPAK Pro versions will follow up within the next few days.

https://sd2snes.de/files/sd2snes_firmware_v1.11.0b2-RevD.zip

Please let me know if this solves your immediate issues.

Again, I’m sorry for the inconvenience and hassle caused.

Firmware v1.11.0 beta 1

Firmware v1.11.0 beta 1

At long last I am starting to feel confident about this release. Due to the amount of features added and since a lot of time passed since the release of v1.10.3 I figured it might be wise to do a beta version first.

As such, use at your own peril. πŸ˜‰

Download here: https://sd2snes.de/files/sd2snes_firmware_v1.11.0b1.zip

Features added:

  • [All] USB (usb2snes) support by RedGuy
  • [All] save state support (via USB/savestate2snes) by RedGuy
  • [All] stand-alone save state features by FURiOUS
  • [All] some compatibility improvements and optimizations for stand-alone save states by ikari_01
    • Add support for YAML list items for multiple savestate_fixes entries per game
    • Add support for simple bitwise operations on savestate_fixes patches
    • Add support for verbatim code execution on savestate_fixes patches – notably fixes Star Ocean (decompressed) save states (however the save state hook still needs CPU time optimization to prevent music glitches in Star Ocean.)
    • Add IRQ support on top of NMI for the save state hook – this enables save states on Out Of This World and possibly other games that only use IRQ. Also fixes controller input capture on a number of games so the game-specific input hacks could be eliminated.
    • Moved save state code outside of the USB hook area to make room for USB hook execution.
  • [All] SGB support by Redacted173.
  • [All] favorite games list by freelancer42

Fixes:

  • [All] Fixed data caching bug in MSU1 data extension which could cause wrong data to be streamed
  • [All] Savestates: do not try to capture data that is already in cartridge space anyway (e.g. SRAM). Fixes text distortion in Near’s Bahamut Lagoon translation when loading a state.
  • [All] Fixed YAML config parser to support comment signs after list start items
  • [All] Fixed SNES open-bus contamination (fixes sprite flickering in Near’s Bahamut Lagoon translation, certain speed runs that rely on open bus behaviour)
  • [All] (hopefully) fixed rapid data line toggling sometimes resulting in address line glitches, causing random crashes in extreme cases, and occasional garbled graphics on SMRPG among others
  • [All] Fixed data integrity problems when loading BSX games

Usage / Testing Notes:

SGB

Note: SGB support requires at least two additional supplementary files:

  • sgb#_boot.bin: The internal boot ROM of the SGB-CPU
  • sgb#_snes.bin: the SGB BIOS ROM that runs on the SNES when you use the SGB.
  • ‘#’ in the filename is replaced with a 1 or 2 depending on the BIOS version you select in the SGB Settings menu, e.g. if you select BIOS version 1 it will look for sgb1_boot.bin / sgb1_snes.bin.
  • The status of the required files can be viewed on the System Information screen.
  • SGB In-game hooks and save states only work when the files match known samples.
  • Please also refer to README.SGB.txt in the sd2snes folder.

Save States

Important notes:

  1. Save states are currently only available on games that do not use expansion chips.
  2. Save states are not currently expected to be fully compatible with 100% of games. If you find a game that does not work with save states, please report – I will be working to improve compatibility.
  3. Save states do not include the state of the SNES’s audio subsystem (APU) due to technical restrictions of the SNES hardware design. Therefore music, sound effects, etc. will remain unchanged after loading a state. Some games maintain close synchronisation between game logic and APU state, requiring (usually) a 1-byte patch to bring them up to date with the APU. These patches are maintained in the file savestate_fixes.yml. For anyone proficient in debugging SNES games who wants to add their own patches, possible formats for patches are outlined in the file.
  4. If you succeed in making a save state fix patch for a new game, please share them πŸ™‚
  5. To use standalone (non-USB) in-game save states, “In-game hook” must be enabled in the “In-game Settings” menu, and “In-game savestates” must be enabled in the “Savestates Settings” menu.
  6. Please also refer to README.Savestates.FURiOUS.md in the sd2snes folder.

When the “Savestate slots” option is enabled, one of four save state slots can be selected in-game using a button combination.

“Load delay (frames)” is used to set a delay after loading a save state before returning control to the game. This gives you some time to let go of the save state buttons and get your fingers back into the normal playing position.

Save states can be saved and loaded in-game using button combinations on controller 1:

ButtonsFunction
Start+LLoad state
Start+RSave state
Select+D-PadSelect save state slot

Using the D-pad one of four save state slots may be selected (Default: Slot 1):

ButtonsSlot number
Select+UpSlot 1
Select+RightSlot 2
Select+DownSlot 3
Select+LeftSlot 4

Favorites List

The favorites list can be viewed from the Main Menu (X button).

To add a game to the favorites list, simply navigate to it in the file browser and press Y to pull up a context menu that displays actions that can be performed on the current selection. (Right now this menu only contains a single entry – “Add to favorites”.

To remove a game from the favorites list, open the “Favorite games” list from the Main Menu, select the entry to remove, and again use the context menu (Y button) and select “Remove from favorites”.

Reporting issues

Please report any issues in the comment section, on Github, Discord (#fxpak-general), or Twitter (@orzvektor).


On a personal note

I regret to say that this release is overshadowed by a recent tragic event.

Near (formerly known as byuu) has passed away on June 27, 2021. To this day I cannot grasp the reality of it all.

On top of his outstanding contributions to the emulation and ROM hacking scene, Near was always a source of inspiration and creativity. Although there were not many occasions on which we worked together, whenever we did it was a blast and put a smile on my face. Near came up with the MSU-1 spec, I made the first hardware implementation of it – thereby turning the sd2snes into a marketable design in the first place. I made a custom SuperCIC specifically for his cartridge dumping project back in 2010. We discovered and got to the bottom of various SNES hardware quirks together. “Professional” endeavours aside, he also encouraged me in getting to know myself better.

I owe Near a lot, and wish I could have thanked him in person one day. I will miss him. A lot.

To honor Near’s memory this release contains a very brief farewell message that will be shown for a few seconds on first boot only (provided that your SD card is not write protected). I hope that you find this tolerable; it was important to me.

Status update – important stability fixes

Status update – important stability fixes

Oof! Well, looks like I couldn’t help but take a deep dive down the rabbit hole after all, staring at scope and LA traces for weeks to find the root causes for some random glitches and freezes. And successfully at that.

Exhibit 1: The Open Bus Glitch

S-PPU1 sprite fetching glitches caused by incorrect latching of written data, triggered by unintended open-bus alteration from the FXPAK. Pixels are flickering in front of the gate near the middle of the screen, occasionally cutting into the dragon’s wings.
Sprite glitch test by undisbeliever, making the issue more pronounced

Hiding in the dark places between CPU cycles this should have no effect on normal operation (except when relying on certain open bus behaviour). What it does is put a value on the bus just inbetween two cycles so no one should ever notice… only to strike when you least expect it!

This bug was first brought up in a different form by the TASBot community and I’ve gotten some tremendously helpful details and troubleshooting from them. Those people really push the technical boundaries to make miracles happen.

Here’s the lowdown: Should you decide to innocently write to PPU register $2100 to change the brightness mid-screen, something nasty might happen. PPUs latch the written value on EVERY 21MHz tick as long as the write cycle is ongoing, not just at the end of it. So it will ALWAYS latch invalid data appearing at the beginning of a cycle, before it can turn valid. In case of forced blanking, this will turn off sprite fetching for just a bit but enough to cause image corruption.

This is an SNES bug at the core – and one you might only ever notice while trying to work around another serious SNES DMA bug like Near did – HDMA to $2100 can break the DMA engine on S-CPU A – but that’s a story for another day.

Just do something mundane like “lda.b #$0f : sta.l $802100” somewhere mid-screen during HBlank and see what happens. The “$80” from the last byte of the second instruction will carry over to the beginning of the PPU write, briefly turning on forced blanking until the CPU asserts the value “$0f” on the bus.

Incorrect data values between HDMA cycles, bamboozling the PPU

The sd2snes/FXPAK just has the “ability” to make this glitch appear even though your data would not normally have the forced blanking bit set at all, by sometimes leaking RAM data on the bus that is not intended for it. This happens because it uses the CPU clock signal’s falling edge to determine that the SNES cycle is over and starts using the RAM for other things like save state or SRAM monitoring. However, since the READ signal is still active it will still output whatever comes from RAM to the SNES bus!

It is a specialty of SNES DMA access timing that the A-Bus READ pulse is substantially longer than the CPU clock active pulse; therefore data from any subsequent use of the RAM can be put on the bus as long as the SNES READ signal is still active.

Sometimes the data output is then turned off during data line toggles, resulting in funny in-between levels like this one:

Data change and slightly late bus cut-off results in an intermediate “frozen” edge (see the green trace near the center not quite reaching the bottom level); digital data changes from $0F to something unintelligible as well

This is something you wouldn’t catch using just a logic analyzer.

The issue was eventually fixed by generating a combined “SNES_PULSE” signal that will stay active until the end of whatever is the longest active pulse so RAM is never cut off as long as the data bus output is active.

Nice and clean.

Exhibit 2: The Coupling Chain Reaction

Causing all sorts of crashes and glitches, but sort of unpredictable and sporadic; e.g. graphical glitches in various games, such as Super Mario RPG or Another World, or random crashes.

It rarely happens but during testing and fixing the timing for the Open Bus Glitch I happened to unmask the effect so it happened much more often – which is a good thing because I was finally able to pinpoint it.

The cause is not unlike the Open Bus Glitch but it happens at the beginning of the cycle, not at the end, and its effect is different.
At the beginning of a read cycle the address has typically been valid long enough for the RAM data to have settled. But sometimes the data is not valid yet because the RAM read might be postponed a bit because an FXPAK-internal access is served (as above, e.g. for SRAM monitoring). In this case the previous value from RAM is put on the bus briefly before the correct one appears. In some cases this causes rapid toggling of all eight data bus lines, depending on the data constellations used by the game.

The rising edges couple into all other lines and momentarily make a wrong address appear on the bus – this in turn causes the FXPAK to fetch an incorrect data byte in the middle of the read cycle. The resulting change in data lines can glitch the other signals again, etc… what the SNES CPU sees is often a mixture of bits from multiple addresses; it then reads a wrong instruction or data, execution flow gets disrupted sooner or later.

In the picture above, Another World / Out Of This World is running and executing at $80E513-$80E514:

The code executed in the oscilloscope trace above, highlighted in green

You can see the last byte of the “lda $0260, x” instruction to the far left of the oscilloscope trace (“02”, near the “B1>” label). The SNES CPU then executes the actual instruction, fetching two bytes from SNES WRAM ($800260) which happen to be the value $FFFF.

Things go awry on fetching the following instruction byte: after READ (yellow trace) goes low, all data lines (the 8 teal colored traces at the top) momentarily go down at once, then quickly up again. The coupling becomes easily visible in the other traces (READ, WRITE, and ROMSEL which is part of address decoding on the SNES); all of them show a significant spike before returning to normal. The FXPAK also sees these spikes and acts upon them accordingly, putting incorrect values on the DATA bus.

The CPU reads the mixture of data bits as $DB instead of $C9 – turning the harmless CMP instruction into an STP instruction which halts the CPU immediately! There is no more activity after that fetch, and the data bus is left floating – you can see the open bus voltage gradually decreasing toward the right end (D0, green trace). The game has crashed and the CPU is essentially dead until a system reset is performed.

For now this issue is fixed by enabling the data bus only when it is safe to assume that the RAM value has settled and any data line toggling is masked from the SNES data bus.

There is room for improvement here. Currently the RAM-to-SNES data path, including address mapping, bus enable and direction switching, is end-to-end combinatorial. An ideal fix for both of these issues would be a partial rewrite of the memory sharing mechanism so it actually buffers RAM data for the SNES bus. This would separate RAM access timing from the actual SNES bus timing with the added benefit of having more free time for shared RAM access.

Progress on current firmware release

Both of the issues mentioned above have been fixed for the base FPGA configuration only so far.

I made improvements to the save state fix data format so it can support more flexible fixes; this was required to make Star Ocean (uncompressed) work with save states.

VRAM DMA mirroring didn’t work properly depending on the source of the data – it didn’t take SRAM into account as a cartridge-based source and tried to sniff the values off the SNES data bus. This led to corrupted party stats in Near’s Bahamut Lagoon translation when loading a save state and has been fixed.

Saving / loading states has been refined a bit for MSU1 so it mutes the audio while accessing the SD card and doesn’t produce a buzzing sound.

I fixed the YAML parser so it can handle multi-line lists with comments in the first line which is good for clarity in the savestate fixes YAML file for games that need multiple fixes.

Next steps:

  • The fixed bus timings now need to be ported to all the other cores (DSP, SuperFX, SA-1, etc.) which should take 2-3 hours of work (and 3-4 days of Xilinx compile time. 😅)
  • Perform some testing on USB features to make sure everything is basically working
  • Check why cheats might not be working (has been reported as an issue)
  • Release a beta πŸ˜‰
Firmware v1.10.3 – REVOKED – DO NOT USE

Firmware v1.10.3 – REVOKED – DO NOT USE

I removed the v1.10.3 download for the time being. It has been observed to overwrite saved games with garbage on SuperFX and SA-1.

For now I recommend to keep using – or revert to – v1.10.1 (on Mk.2) or v1.10.2 (on Pro).

Sorry for the hassle.

Firmware v1.10.3 released

Firmware v1.10.3 released

Firmware 1.10.3 is out with critical fixes for sd2snes Mk.II.

Download here!

What’s new:

  • Fixes:
    • [Mk2] Revert S-DD1 bitfile to earlier build which happens to less expose a timing hazard. This will need further examination for a proper fix. Fixes severe glitches with S-DD1.
    • [All] Fix swapped logic terms in SA-1 and SuperFX RAM write cycles. Fixes severe glitches on SA-1 and SuperFX on Mk.II units; The bug was also present in the Pro firmware but didn’t seem to have much of an effect.
    • [All] Fix game video mode setting in SuperCIC pair mode.
Firmware v1.10.2 released

Firmware v1.10.2 released

Firmware 1.10.2 is out with a round of fixes.

Download here!

What’s new:

  • Fixes:
    • [Pro] fix wrong data ROM size on uPD96050 core -> F1 ROC II track is shown properly.
    • [Pro] fix S-DD1 address mirroring -> Star Ocean works
    • [Pro] correct Cx4 core speed from 96MHz to 80MHz
    • [All] apply volume boost after FPGA reconfiguration (fixes MSU1 volume boost on Cx4, OBC1, SuperFX, SA-1, S-DD1)
    • [All] fix S-DD1 writes going to the wrong address on concurrent MCU accesses (saving occasionally not working in Star Ocean)
    • [All] avoid potential data corruption on MCU writes (e.g. SPC loading)
    • [All] Workaround for Super Nt not booting when CIC enters pair mode successfully