The benefit of this is that it takes no set-up for non tech users. The major downsides are that it's only 2 player, and only works for midi instruments.
On Fourhands you can actually achieve very low latency (<20ms) with WebRTC for somewhat close players on wired connections. If you're having latency trouble like your readme says, I'm guessing it's not the WebRTC part.
I'm getting <20ms with jamulus on a usb audio card on ubuntu studio (rt-linux). This is for a weekly rehearsal with a big-band. The server runs on an ODROID in the house of one of the players. Connection is with ethernet and glass fiber.
We previously talked about JackTrip WebRTC[1], which I first thought was similar. But while JamRTC uses WebRTC, it's not actually browser based, while JackTrip WebRTC is. (And of course they are similar in that they are tring to do low-latency audio conferencing via WebRTC.)
And then there is regular JackTrip, which the author refers to in the FAQ, which is neither browser- nor WebRTC-based.
I worked on a project similar when the pandemic started last year. Ran into latency problem and just couldn't find a way around it. Add to the latency issue is also the global metronome as well.
Wow neat! I’ve been working on something similar, just optimized for keyboards only with midi data, intended for use as a complement of having a remote teacher on the line while using it.
The project is called Midishare https://github.com/corytheboyd/midishare. The site is live but I am very much working on core functionality still. If you do want to try it just reach out, or hop on the discussion forum linked in the nav!
Awesome stuff. If you use it, do yourself a latency favor and avoid WiFi connections. Use ethernet.
Some SDP munging in the clients can shorten Opus packets, which helps. If they're longer than 20ms things get ugly.
I tried using a swarm approach (full mesh of point-to-point WebRTC connections) for plain audio, no video. I got the latency down to about 70 ms transcontinental, but my musician alpha tester (brother) didn't go for it.
My experiences playing with Jamulus during our lockdown follow.
Wired networking has less latency than wireless.
Sounds card and drivers add to the latency mix. Which is why Jamulus requires ASIO drivers (on windows at least) - the inbuilt mic on your laptop does a heap of processing to clean up etc, but adds to latency; unfortunately, if you remove this processing there's a lot of noise. A decent external sound card is pretty much required.
If you can isolate the sound of your instrument so you only hear it via the network (i.e. you're wearing isolating headphones), you can get away with more latency as your brain will compensate. That way, everyone will listening through the network, so everyone should be in sync (assuming the software compensates for different latencies). But this makes things really hard for vocalists or drummers, where isolation is really hard or impossible.
The upshot is that this is really hard for amateurs - I was trying to get something going with my community orchestra but gave up as half of them were on ipads or didn't know what an ethernet cable was, let alone an external sound card. Was tempted to look into building a raspberry-pi based standalone 'jamming' applicance, but then the lockdown ended and I lost interest. Still, exploring public Jamulus servers felt a bit like the early days of the Internet :)
I plated with WebMIDI a few years ago. Thought it would be fun to build a player piano broadcast service, where anyone with a MIDI device could play a piano remotely. Never built it though.
We are layering AI on top of all kinds of problems. It feels like the sync problem when latency is introduced is something that you could solve with some dumb AI.
Is AI solving sync part of the third wave for musicians composing over the internet in real time?
Well, I'm thinking that AI could observe the sync issues, determine what the right thing is (based on patterns from other prior performances), and then adjust dynamically. It won't be perfect but it might have enough syncopation to make interesting music.
I've been jamming with some friends on Jamulus. A complicating factor is that the effect of latency depends on what instrument you play. When I play an electric instrument (bass guitar) that makes little or no sound of its own, then it's easy to monitor my timing in the mix through headphones, and my fingers just adapt by playing ahead of the beat as needed. Our bodies have to do that anyway in order to play in time because initiating a note happens before the note comes out of virtually any instrument.
It's much harder when you don't have that isolation, for instance when I play an acoustic instrument, I have to turn up my phones to drown out the acoustic sound. And it gets still harder if you hear that instrument by bone conduction, e.g., horns or voice. The band adapts by the "easy" instruments taking charge of the tempo and letting the other instruments float around a bit. Still, it takes a lot of concentration to avoid getting lost, and just doesn't sound as good as a live performance of a "tight" band.
The band members have agreed that it's better than not playing at all, but might not be a good listening experience for an audience. Not necessarily because the delays are perceptible but because it sounds like the band is struggling to stay together. Of course other things are happening as well, such as lack of visual cues.
Fortunately we're all "of a certain age," and between the weather warming up and us getting vaccinated, we'll be playing together again.
A marching band might span a whole American football field during a performance. That's potentially 135ms to the center of the field from the edge or more from a corner. Their type of music may be somewhat designed around it as a limitation as well as having a visual metronome in the conductor (though you could estimate one and even make it audible instead of visual using half round-trip-time for internet jam sessions).
People seated at near field level at opposite corners may hear a 350ms+ difference in some instruments, but they may spread them out so that it just sounds like a reverb/wash of echoes.
For these kind of apps wear headphones and you cut around 3ms per meter you are away from your speakers, due to speed of sound.
There's the speed of light on one hand and a maximum latency of roughly 30 ms, which is the boundary below which the human mind perceives sound as synchronous, on the other.
However, neither does prohibit non-colocated live music altogether.
Keep in mind the threshold is much lower when playing an instrument is involved. For electric guitar, for example, I've found anything above 13ms is perceptible and makes playing in time more difficult.
jamkazam (https://jamkazam.com/) is commercial product that does this. They also make custom hardware optimized for latency and streaming. I used it a few years ago (before the custom hardware was available) and getting all the members' hardware configured was a bit of a pain, but the potential was there.
https://github.com/jminjie/fourhands
The benefit of this is that it takes no set-up for non tech users. The major downsides are that it's only 2 player, and only works for midi instruments.
On Fourhands you can actually achieve very low latency (<20ms) with WebRTC for somewhat close players on wired connections. If you're having latency trouble like your readme says, I'm guessing it's not the WebRTC part.