Jeff Kaufman's Writing
Jeff Kaufman's Writing订阅

Rhythm Stage Setup v5

I recently finished the largest set of changes I've made to my rhythm stage setup in 4.5y. The basic interface is similar, but I have a lot more flexibility, more control, more options, and it's easier to travel with. My favorite part is the speech recognition for calling chords.

I started off playing mandolin and piano, and then over the years added progressively more expressive footdrums. I added a breath controller. I built out a complex but expressive system where these different interfaces could interact to let me produce a coherent rhythm section. After a frustrating experience moving from one Mac to another and wanting to simplify, I moved from the Mac to a Raspberry Pi, set it all up as a pedalboard, and then again as a lighter pedalboard.

During the last year and a bit, however, I haven't been tinkering. It did what I wanted, in a reliable and portable fashion. And it was hard to modify: the pedalboard was static, and the Raspberry Pi code wasn't easy to get at either. Which was fine, and probably good for me! I just played a lot of gigs (23 Kingfisher, 3 Free Raisins, 3 Dandelion) and got better at it.

But then it started breaking. My whistle synth box didn't always come up, the drums sometimes needed an external DI, and a USB sound card snapped. Flying back from Atlanta my gear suffered rough enough treatment to bend a USB B connector, and it was clear my system needed some love.

I started with the most urgent portion: the whistle synth. I had Claude port it over to my laptop, where I turned it into a Mac app. This also let me add several more voices, and a mute option that never made loud clicks (I'm not really a hardware person).

This project made me realize that AI getting capable meant that the tradeoffs around laptop-vs-embedded had shifted. I could have everything on my laptop, and without needing to put painfully large amounts of time into programming it could all be fully defined in code instead of manually-configured GUIs. Having it on my laptop would make it easy to apply models to modify and improve it, and open up a lot more lexibility.

I decided to move as much as possible over to the laptop. This meant no more Raspberry Pi, whistle synth box, physical switches for signal routing, and computer code instead of guitar pedals. I moved from:

To:

Four fewer pedals, three fewer DIs, two fewer power bricks, and no little computers. Saves ten pounds: 32lb to 22lb. The Mac weighs ~3.5lb, but I'm traveling with it regardless.

Of these savings, 3lb (2.7lb of talkbox + 0.3lb of power supply) was from giving up the talkbox. I liked how this simplified my setup, including less complex audio routing, but after playing Moondance both Cecilia and I really missed it. So I added it back:

A large benefit of moving to the Mac is I can now see what I'm doing. I control my rig with an external keyboard, covered in little labels that remind me what each key does:

For example, before playing Trip to Moscow I would probably press FB (footbass), S (make footbass play synth bass), FX (breath-controlled keyboard overlay), set the drums to custom kit 63 (electronic kit), and then press L (keyboard velocity-insensitive bass overlay) and change the keyboard to epiano2 when we got all electronic. And press JH (breath-controlled bass-following low bass) at some point partway through for more ability to pulse the offbeat. Without a monitor, I needed to keep this all clear in my head, which was especially important since many settings are modal or interact. I usually didn't mess up, but partly because I tried not to dial in anything with too many presses.

Now I can just represent the state on my monitor:

This is also very helpful for showing which voices apply to which instruments. Previously I needed to remember that on FX (breath-controlled overlay) the B key would give me a supersaw, while on Dc (chord drone) the B key was a different voice (I forget what). Now I can choose a set of voices for each instrument that are the ones that sound best with this kind of control, without worrying about how I'll remember which button gives me which sound.

This is basically all "what I had before, but easier to control". Which is good, but the real payoff comes from making it much easier to extend the system. I've done a lot here, with extensive help from Claude Code, and it's going to take me a bunch of time to learn how to play these new options well.

A big one is that because I now route my mandolin through the system I can replace pedals with code, add more effects than it was worth carrying pedals for, and write effects that consider what other things in the system are doing. This includes using the breath controller as an expression pedal, or tonal effects that care about the active chord. Demo video:

youtube

I also added speech recognition, where I can turn things on and off vocally, and even call chords. For chord calling I had Claude make me a custom speech recognizing system for the numbers one through seven (Nashville numbers) trained specifically on my voice and optimized for minimizing latency (decision in 40-170ms). It also runs Apple's in parallel, primarily to allow learning from mistakes but also to handle numbers the custom one misses and any non-number commands. Here's a video from earlier today, showing how I was still working to improve both my call timing and the code that triggers the changes; you can hear that many changes are off by a beat:

youtube

Here's a short updated video after a bunch of work this afternoon and evening to make number calling more reliable (af196263f). This involved adding a prior that chords run for a multiple of 2 beats on a grid implied by previous called chords, better word boundary identification, and a bunch of parameter tuning where I made recordings as I worked with it. I'm much happier with it now, and the two mis-timed changes in the video below are me calling early (0:59, 1:15):

youtube

I'm also trying to figure out new things I can do with the breath controller, and one thing I've been playing around with is percussion:

youtube

Other stuff:

  • Vocal effects, including ones like a vocoder that care about the currently active chord. Not sure I like any of these enough yet to use live.
  • Cecilia needs to hear my mandolin playing, but gets plenty of computer overlays in the monitor wash already. This was a problem with talkbox, which can be fed by either the mandolin or computer. Now that it's all running through my box, though, I can make talkbox go to the mandolin output channel if mandolin is included, and to the computer output channel otherwise.
  • I auditioned and added a lot more voices. Including adjusting which voices are available for which instrument by replacing some that didn't work with that instrument's method of articulation. I don't know if any of them are worth it yet, though: I still mostly do the small number of things I like.
  • I added French Canadian feet back into the system, after discarding them when I switched to the Raspberry Pi. They're no longer samples, but synthesized, which lets me use how hard I stomp or the breath controller to adjust how hard the hit is. I like these a lot, and used them live to good effect at the one gig I've done since this change.
  • I also added a "stompy feet" option which is feet with a bassier kick, a "bare feet" option with sounds synthesized to imitate my own bare feet, and a "hand drums" option which was a Claude "happy accident" that sounded interesting enough to keep. I'm not sure I'd actually use either of the second two, though.
  • When used stand-alone, the Focusrite 2i2 takes in a mic (whistle bass, calling chords) and mandolin and gives two channels (computer and mandolin + feet, though really it's "doesn't go to Cecilia's monitor" and "does go to Cecilia's monitor"). I've also added integration for my custom 1818VSL mixer, in which case you skip the 2i2, run the mic and mandolin into channels 1 and 2, and all my processing happens automatically before the mixer sees channels 1 and 2. Kind of an automatic effects loop.
  • I added recording to the custom VSL mixer, including logging MIDI signals and keyboard presses so it's easier to evaluate later (or tweak if we wanted to turn recordings into another live CD).
  • I'm toying with starting to run our own monitors by default with the VSL, since it has enough output channels to pass my normal outputs to the sound board and then also give a mix to each of us. I'd use XLR Y-cables on the other inputs so the board would also get a full copy of everything.
  • Outputs can now be stereo, in case we wanted to switch to in-ear monitors. But I really like stage wedges for feeling connected to the dancers: being responsive to them in the moment is, to me, a big part of the appeal of playing live for dancing. And it would be a fourth thing connected to my head (after the mic, talkbox tube, and breath controller) making me even more borg.
  • My keyboard (Yamaha P-121) includes USB audio support, and I already have it connected to my computer over USB for MIDI purposes. Using this stream avoids several lossy steps, and I've made the VSL mixer bring it on as an "input 9" when available. Sadly if I'm ever working with someone's P-125a keyboard, the "a" means that they dropped the USB audio feature.
  • I added a bunch of features with technocontra in mind (kick ducks, pads, weird effects) but I don't think I like any of them even for technocontra. Not sure yet.
  • I tried a version where whistling picks a chord, but didn't end up liking it at all.
  • One cool thing about a whistle-controlled synthesizer and speech recognition is that I can do both of them at once on the same mic: they're very easy to tell apart.
  • Overall, I'm still mostly playing live with only a small portion of the possibility space I've built for myself with this instrument. Some of this is that a lot of things that are possible turn out not to be very effective in a noisy dance hall, or just don't sound good anywhere. Some of this is that there are some very neat things that it's possible to do that I haven't practiced well enough to take advantage of. And some is that I just forget to try some things live.
  • When I switch to pedals choosing bass instead of being drums it now switches to a preset that's a better fit for that, so I press one button ("DCN": drum chooses note) instead of several ("DCN", "FB", "D", "v", "+", "+", "+", oct up).

This post took me way too long to get out, both because I keep adding to the system and because I'd rather make demo videos once I'm good at controlling the system. Since that kept resulting in not publishing, I've lowered my threshold to get something out. I can always follow up as I do more, and several of the ideas above would have been better as their own posts.

The code is all on github if you or your agent would like to look:

Comment via: facebook, mastodon, bluesky

参与讨论