No posts yet. Check back soon!
Trying Out the New VBCC-65816 Compiler
07:30 PM · Oct 10, 2026
At some point since the last time I looked, the VBCC/Vasm/Vlink suite I generally recommend for Amiga and Atari ST development has also gained support for the 65816 CPU, with both SNES and Apple IIgs targets. That’s exciting, because GS/OS’s native Object Module Format for its executables are used nowhere else—until this point I could only really offer Merlin 32 for producing these files for free on a modern system. Let’s take it for a spin and see how it does!
Bottom line up front: this is clearly still a work in progress, and it acknowledges itself as such in its documentation, but it is in fact in good enough shape already that pure-C code is usable. It does not yet have anything like parity with its m68k editions.
Picking a Test Case
The very first thing I ever did with VBCC was point it at a command-line program (an proof-of-concept from the Simulated Evolution project and build it to run on the Atari ST’s EmuCON console:

This program does 64-bit math and relies on several C99 features, so it was a very solid test, and the m68k VBCC systems passed it with flying colors. I only needed to make one modification: the simulation state was about 60KB, and VBCC’s code generator for the m68k objects rather strongly to stack frames being over 32KB. This was really a flaw in the original code, using 32-bit integers where 8 bits would do, and fixing that dropped the stack usage to a more comfortable 16KB.
That still wasn’t enough to survive on the 65816, though. Its stack indexing only gives us 256 bytes. I had to move the simulation state into a proper global variable, where it probably should have been in the first place.
Furthermore, it looks like the standard libraries are still incomplete at time of writing and the 64-bit math operations aren’t in yet. I fixed that by bringing in the simpler PRNG from my 16-bit DOS edition.
With those two changes, we’re in good shape!

… well, at least for awhile, and then things go awry:

It looks like int is 16 bits on this platform. This, along with the lack of 64-bit integers, puts us in a space that looks a lot more like the early DOS world than it does with the Mac, Amiga, or Atari ST. We even have to contend a bit with “tiny” vs “large” vs “huge” memory models, since we need to deal with bank registers and such. This, too, is incomplete in vbcc, but there’s enough there for our purposes.
Being restricted to DOS isn’t going to really slow us down anyway, because the DOS port was based on the same C engine core. We just need to write a new wrapper around it if we want a fully graphical version.
Porting Simulated Evolution to the IIgs
There’s a bunch of little parts here, but we can treat them in isolation. We can also stay entirely in C for the whole program; the one system call we need is actually provided by vbcc‘s libraries and everything else can be done with memory-mapped I/O or truly portable code.
The Random Number Generator
This is the 16-bit xorshift RNG that I’ve been using for ten years.
static unsigned short rng_x = 1;static unsigned short rng_y = 1;static void seed_rng(unsigned long seed){ rng_x = (unsigned short)seed; rng_y = (unsigned short)(seed >> 16); rng_x |= 1; rng_y |= 1;}static unsigned short rng(void){ unsigned short t = (rng_x ^ (rng_x << 5)); rng_x = rng_y; return rng_y = (rng_y ^ (rng_y >> 1)) ^ (t ^ (t >> 3));}
Keyboard I/O
This works broadly the same way as it does everywhere on the Apple II platform: $C000 has its high bit set if a new key has been pressed, the low 7 bits are the key that was pressed, and we acknowledge the keypress by reading $C010. The one concession we make to C and the IIgs is that we use the full 24-bit addresses for them in their true location on page $E0:
static int inkey(void){ unsigned char c = *((__far volatile unsigned char *)0xe0c000); if (c < 0x80) return 0; /* acknowledge keystroke */ unsigned char ignored = *((__far volatile unsigned char *)0xe0c010); return c & 0x7f;}
Graphical Output
We enter and leave super-high-res mode by setting or clearing the top two bytes of $E0C029. The graphics themselves are in $E12000–$E19FFF as described in my old article on super high-res mode. In addition to changing the mode, I also draw the big blue rectangle that is the primordial soup, store the original border color, and then set that color to black. We restore the border once we return to text mode.
__far uint8_t * const shr_screen = (__far uint8_t * const)0xe12000;__far volatile uint8_t * const border_reg = (__far volatile uint8_t * const)0xe0c034;static uint16_t palettes[4] = { 0x0000, 0x000a, 0x00a0, 0x0fff }; static uint8_t border_col;void graphics_mode(void){ __far uint16_t * const word_screen = (__far uint16_t * const)shr_screen; __far uint16_t * const shr_palette = (__far uint16_t * const)0xe19e00; for (int i = 0; i < 80; ++i) word_screen[i] = 0x1111; for (int i = 0; i < 5; ++i) shr_screen[i] = shr_screen[i+155] = 0; for (int i = 80; i < 16000; ++i) word_screen[i] = word_screen[i-80]; for (int i = 16000; i < 16384; ++i) word_screen[i] = 0; for (int i = 0; i < 4; ++i) shr_palette[i] = palettes[i]; *((volatile unsigned char *)0xe0c029) |= 0xc0; border_col = (*border_reg) & 0x0f; (*border_reg) &= 0xf0;}void text_mode(void){ *((volatile unsigned char *)0xe0c029) &= 0x3f; *border_reg |= border_col;}
Actually putting pixels on the screen is pretty easy, but because the 65816 has no multiplier I try to do everything in terms of bit shifts. Since we’re dealing with “double-sized pixels”, the default 4-bits-per-pixel mode turns out to be really convenient because we’ll be updating the graphics a full byte at a time. We just have to write the value to the target row and the row just below it.
static void pset(unsigned int x, unsigned int y, unsigned char c){ /* offset is y * 320 + x + 5, but 65816 is bad at multiplication so we * do it with nested shifts */ /* y * 320 = (y * 5) * 64 */ unsigned int i = (((y << 2) + y) << 6) + x + 5; shr_screen[i] = c; shr_screen[i+160] = c;}
Callbacks From the Simulation Core
There are a handful of routines we need to supply that the simulation core demands of us.
The first is rand_int(n), which gives us a random number between 0 and n-1. Unlike the 64-bit PRNG I used in the m68k versions, the RNG here is less random in its low bits and we are obliged to multiply the raw return value and then extract the high bits from the 32-bit result, like so:
unsigned long rand_int(unsigned long n){ unsigned long v = rng(); return (v * n) >> 16;}
Then we have the draw and erase routines, that simply forward to pset.
void erase_bug(int x, int y){ int dx, dy; for (dy = 0; dy < 3; ++dy) { for (dx = 0; dx < 3; ++dx) { pset(x+dx, y+dy, 0x11); } }}void draw_bug(int x, int y){ int dx, dy; for (dy = 0; dy < 3; ++dy) { for (dx = 0; dx < 3; ++dx) { pset(x+dx, y+dy, 0x33); } }}void draw_plankton(int x, int y){ pset(x, y, 0x22);}
The callbacks the command-line simulator used are here too, but they’re no-ops.
void report_bug(const evo_state_t *state, int bug_num, const char *action){ /* Callback ignored */}void report_birth(const evo_state_t *state, int parent, int child_1, int child_2){ /* Callback ignored */}
The Main Loop
The main loop isn’t much to write home about; it’s broadly the same as it is on DOS. It runs the cycle, checks the keyboard, and waits for the timer to tick over. GS/OS includes a built-in 60Hz timer that we can check with the GetTick() routine, which vbcc in turn defines a C interface for in <OS/MiscTool.h>.
int main(){ int garden = 1; /* Start out in garden mode */ graphics_mode(); seed_rng(GetTick()); /* Init PRNG */ initialize(&state); /* Init simulation */ while (1) { unsigned long start_time = GetTick(); int c; run_cycle(&state); if (garden) { seed_garden(&state); } /* User input: toggle garden on 'g', quit on ESC */ c = inkey(); if (c) { if (c == 'g' || c == 'G') { garden = !garden; } if (c == 27) { break; } } /* Advance no faster than 60Hz. */ while (GetTick() == start_time); } text_mode(); return 0;}
The Apple IIgs Toolbox Reference suggests that the “Miscellaneous Tool Set” that supplies this function needs to other libraries prepared and for a VBLANK interrupt to be configured; either the overall OS does this for us going in, or vbcc is handling any necessary setup as part of its pre-main startup work. The function works fine as written above.
Results
The results look quite a bit like the Atari ST build, unsurprisingly.

The final program weighs in at 13KB. Interestingly, my initial Hello World program also weighed in at 13KB. The command-line test I kicked off this article was 32KB, much larger than the graphical version.
I guess the main moral of the story here is that stdio.h and printf are really, really expensive.
The other moral would be that portable C ends up being very portable indeed: this core simulation code is now shared, with no changes whatsoever between seven platforms: SDL2, D3D9, Amiga, Classic MacOS (Systems 4 through 9), Atari ST, DOS, and Apple IIgs. Only the application shell changes, and, as we’ve seen above, that shell can be pretty small.
Dipping Into Assembly Language
Both the Amiga and Atari ST ports ended up providing a little platform library to handle things like OS interoperation in cases where the C layer didn’t really offer everything I wanted. (For Amiga, I had set myself the specific goal of never directly relying on the NDK, which in retrospect was a mistake, but which certainly gave me a lot of practice!) I’d have liked to do that here, but after looking at vbcc’s assembly language output, I shied away for the moment.
Everything clearly works, mind you, but the documentation was thin enough, and the behavior of vlink unclear enough, that I felt I was poking at something whose form had not quite yet solidified. While the large memory model seems complete enough that I’d have no trouble interfacing with it (all calls and pointers are long mode, all registers are 16-bit at all function boundaries, arguments passed on the stack with the first integer argument in .AX, and return value in .AX when possible and via secret argument when not in the C style) what the compiler said it was doing didn’t really line up with what I was seeing in the final binaries.
In particular, sections all had markers like this:
; stacksize=0+??
;vcprmin=10000
section "DONTMERGE_text.far.rng.7","acrx"
One might expect this to mean that the sections wouldn’t merge. However, looking at the final OMF file reveals that, in fact, all the code in the program was merged into a single segment named 000000000_text.startup. The data, on the other hand, seemed to be split up quite heavily, with three main-memory data segments out of two translation units. Perhaps some of it was coming in from libraries.
In the end I decided that I was better off just relying on the libraries at the moment. Once the memory models and linkage systems are fully designed it should be a more stable foundation upon which to mix and match.
Of course, we don’t have to mix and match…
Using Vasm For Everything
Vasm’s 65816 support is supplied as part of its 6502 support; vbcc’s output is expected to be consumed by the vasm6502_oldstyle assembler in particular. The syntax for this version is very similar to ca65’s, with only a few changes needed to port over a few of my old ProDOS 8 IIgs projects:
- You need to pass the
--dotdirargument to use ca65-style directives; otherwise it wants things likeORGandBYTEto stand alone as pseudo-opcodes. - The
.i8and.i16directives are instead spelledX8andX16. - Vasm doesn’t seem to have smart/follow mode, so you need to redeclare appropriately after every
SEPorREPthat touches the MX flags. - Instead of
a:orf:prefixes to force 16- or 24-bit addressing modes, vasm uses!and>.
ProDOS 8 binaries are just raw binary dumps in the first place, so vasm’s -Fbin output format will produce exactly what you need. My vasm ports produced bit-exact binaries to the ca65 ones.
Overall I think that between ca65, Merlin, and vasm, vasm comes off the worst. It demands the most work to keep register widths straight and I really do not like the overloading of < and > for forcing direct page or far mode, compared to their usual use for low/middle byte. On the other hand, I will also admit that Merlin’s tradition is clearly a very different one independent of the one that both ca65 and vasm share. In all these cases the reason you would be picking Vasm over the others is not for the assembler, though; it’s for the linker.
The ld65 linker that’s part of ca65’s suite is fundamentally unsuited for producing relocatable code in the OMF style. OMF is significantly more like COFF or ELF than the images that ld65 is really tuned for. Meanwhile, Merlin’s linker obliges us to organize our program in a fairly rigid way because of the constraints it inherited from its ancestors. Vlink is fully modern and we can organize our sources and our object files however we please.
I do think, at this point, that it would still be premature to lean on this. OMF is a heavily self-describing format with each loadable section having special caveats and markers to ensure that they only load into particular parts of memory, and it’s also common for the direct page (the 65816’s relocatable zero-page equivalent) to share a segment with the process stack. This is another place where I feel like I want to give it time to fully mature before I really start leaning on its features.
But the features that are already there do indeed show great promise. I’m happy to see a credible competitor to Merlin show up, and even happier to see a credible competitor to the still-commercial ORCA/C compiler.