Discussion:
[Savonet-users] Webcaster: Connecting WebRTC stream to the outbound audio stream
Mark E
2017-02-17 15:08:15 UTC
Permalink
Hi,

First let me say that webcaster is a great tool - thank you very much
for creating it along with webcast.js and liquidsoap.

I would like to integrate a simple WebRTC audio-only chat into the
webcaster client code so that an inbound WebRTC caller is connected to
the webcaster host's (the person operating webcaster) voice audio and
mixed into the outbound audio stream being sent to liquidsoap.

I have the chat built and it works fine. Now I need to integrate that
into the webcaster app. I need some advice please. This is what I think
should be done:

- When the "Start streaming" button is clicked and the app does its
navigator.mediaDevices.getUserMedia() call to get permission for the mic
and the result stream, add code to connect to my WebRTC signaling server..

- On a different web page where people listen to the stream, add a
button to initiate a call to the studio. When the button is clicked the
listener is asked for permission to use the mic and then on success is
connected to the signaling server.

- Signaling server sends an offer to connect to the webcaster host,
which then (after the usual WebRTC negotiation) results in a RTC peer
connection between the show and the caller.

- At that point I need to somehow connect the caller's audio stream
(object) into the overall webcaster audio mix so that the conversation
goes out to liquidsoap as usual.

The question I have is where and how within the webcaster client.js do I
connect the callers stream to the outbound audio stream? I'm not an
expert with backbone and underscore (used with webcaster) but I can wing
it as I learn if someone can please point me to the right things to do.

Thanks for you time in considering this!

Mark
Mark E
2017-02-18 04:50:04 UTC
Permalink
Here's where I am at on this:

After the WebRTC caller connects to the webcaster radio host (which
works fine, the show host and caller can talk to each other) then I do
this (using the webcaster's current AudioContext - so I think anyway -
where event.stream is the caller's audio stream object (taking WebRTC
audio into Web Audio for processing, which is definitely possible) :

source = Webcaster.node.context.createMediaStreamSource(
event.stream );
gain_node = Webcaster.node.context.createGain();
source.connect( gain_node );

// So with the above I have a gain node with the caller's audio stream
connected to it, or so I think.
// Now I connect the gainNode to the Webcaster's node.context.destination

gain_node.connect( Webcaster.node.context.destination );

No errors are returned in the JS console, but the code doesn't cause the
caller's audio stream to be sent out via webcaster to liquidsoap and on
to Icecast (I'm monitoring the audio stream in a different computer via
a MP3 stream player connected to the Icecast stream). The webcaster
host's audio is sent through liquidsoap to Icecast, I can hear that.

What am I doing wrong?

Mark
Post by Mark E
Hi,
First let me say that webcaster is a great tool - thank you very much
for creating it along with webcast.js and liquidsoap.
I would like to integrate a simple WebRTC audio-only chat into the
webcaster client code so that an inbound WebRTC caller is connected to
the webcaster host's (the person operating webcaster) voice audio and
mixed into the outbound audio stream being sent to liquidsoap.
I have the chat built and it works fine. Now I need to integrate that
into the webcaster app. I need some advice please. This is what I think
- When the "Start streaming" button is clicked and the app does its
navigator.mediaDevices.getUserMedia() call to get permission for the mic
and the result stream, add code to connect to my WebRTC signaling server..
- On a different web page where people listen to the stream, add a
button to initiate a call to the studio. When the button is clicked the
listener is asked for permission to use the mic and then on success is
connected to the signaling server.
- Signaling server sends an offer to connect to the webcaster host,
which then (after the usual WebRTC negotiation) results in a RTC peer
connection between the show and the caller.
- At that point I need to somehow connect the caller's audio stream
(object) into the overall webcaster audio mix so that the conversation
goes out to liquidsoap as usual.
The question I have is where and how within the webcaster client.js do I
connect the callers stream to the outbound audio stream? I'm not an
expert with backbone and underscore (used with webcaster) but I can wing
it as I learn if someone can please point me to the right things to do.
Thanks for you time in considering this!
Mark
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Savonet-users mailing list
https://lists.sourceforge.net/lists/listinfo/savonet-users
Romain Beauxis
2017-02-18 16:54:52 UTC
Permalink
This post might be inappropriate. Click to display it.
Mark E
2017-02-18 21:30:03 UTC
Permalink
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Romain Beauxis
2017-02-19 17:31:37 UTC
Permalink
Awesome. Feel free to submit a PR when you're ready.

I want to do the SSL release as soon as possible but I don't think it'll
happen until March or so..

Romain
Great tips, much appreciated. I think I have the basics working -
woohoooo!
I need to put this code out on an Internet accessible dev box (get it off
my localhost) so I can do some testing with more than one caller. Which I
can do. BUT, one small problem: The current version of liquidsoap is 1.2.1
and it doesn't appear to have support for the new input.harbor.ssl inputs.
This presents a problem because Chrome refuses to connect to insecure
non-SSL Web sockets unless it's on a localhost network (127) network.
Can you all please finalize the liquidsoap build for the opam package so
that it has the new SSL features? (RE: Github #376 linked below - January
7, 2017 - "SSL support for harbor", looks like it may be 99% ready. )
https://github.com/savonet/liquidsoap/pull/376
<http://h5ks.mjt.lu/lnk/AEAAHpZTiiEAAAAAAAAAAGgH3uUAASJ-8QAAAAAAAAgArwBYqLzevEDLj7I6TJaVrbRJp9rrAgAHnU0/1/wMJ_1W7aJmah87N_NacWeQ/aHR0cHM6Ly9naXRodWIuY29tL3Nhdm9uZXQvbGlxdWlkc29hcC9wdWxsLzM3Ng>
SSL would a really great enabling feature so that people that need to can
put liquidsoap on a non-localhost network system (e.g. have a totally
web-based web-hosted system).
Anyway when I finish testing and tweaking my Webcaster mods I can share.
BTW: If you guys had a donate button somewhere I'd be more than happy to
send some bux to help get the SSL feature rolled out!
Mark
Hi,
Thanks for your interest! This seems like a nice feature to experiment with!
The code is a bit hairy but so are the various JS Audio APIs.. I'd suggest
that you look at the microphone code, since a WebRTC call in seems pretty
similar to streaming from the microphone..
The code for the model is here: https://github.com/
webcast/webcaster/blob/master/src/models/microphone.coffee
<http://h5ks.mjt.lu/lnk/AEAAHpZTiiEAAAAAAAAAAGgH3uUAASJ-8QAAAAAAAAgArwBYqLzevEDLj7I6TJaVrbRJp9rrAgAHnU0/2/rYTh_6Oi50UEeohpRKcMnA/aHR0cHM6Ly9naXRodWIuY29tL3dlYmNhc3Qvd2ViY2FzdGVyL2Jsb2IvbWFzdGVyL3NyYy9tb2RlbHMvbWljcm9waG9uZS5jb2ZmZWU>
createSource, the audio source is asynchronously created.
A quick path to experiment could be to simply hack on this class and use
the existing microphone UI until you have it working as you wish and then
split it into a new source in the webcaster UI.
Hope that makes sense and can help. Remember to rebuild your coffee script
stuff after doing your changes! `make` would work for that. Also, `make
run` will start a locally served test page at http://localhost:8000/
<http://h5ks.mjt.lu/lnk/AEAAHpZTiiEAAAAAAAAAAGgH3uUAASJ-8QAAAAAAAAgArwBYqLzevEDLj7I6TJaVrbRJp9rrAgAHnU0/3/_z26ESO4hE7rs-_6pB2pCw/aHR0cDovL2xvY2FsaG9zdDo4MDAwLw>
Romain
Post by Mark E
After the WebRTC caller connects to the webcaster radio host (which
works fine, the show host and caller can talk to each other) then I do
this (using the webcaster's current AudioContext - so I think anyway -
where event.stream is the caller's audio stream object (taking WebRTC
source = Webcaster.node.context.createMediaStreamSource(
event.stream );
gain_node = Webcaster.node.context.createGain();
source.connect( gain_node );
// So with the above I have a gain node with the caller's audio stream
connected to it, or so I think.
// Now I connect the gainNode to the Webcaster's node.context.destination
gain_node.connect( Webcaster.node.context.destination );
No errors are returned in the JS console, but the code doesn't cause the
caller's audio stream to be sent out via webcaster to liquidsoap and on
to Icecast (I'm monitoring the audio stream in a different computer via
a MP3 stream player connected to the Icecast stream). The webcaster
host's audio is sent through liquidsoap to Icecast, I can hear that.
What am I doing wrong?
Mark
Post by Mark E
Hi,
First let me say that webcaster is a great tool - thank you very much
for creating it along with webcast.js and liquidsoap.
I would like to integrate a simple WebRTC audio-only chat into the
webcaster client code so that an inbound WebRTC caller is connected to
the webcaster host's (the person operating webcaster) voice audio and
mixed into the outbound audio stream being sent to liquidsoap.
I have the chat built and it works fine. Now I need to integrate that
into the webcaster app. I need some advice please. This is what I think
- When the "Start streaming" button is clicked and the app does its
navigator.mediaDevices.getUserMedia() call to get permission for the
mic
Post by Mark E
and the result stream, add code to connect to my WebRTC signaling
server..
Post by Mark E
- On a different web page where people listen to the stream, add a
button to initiate a call to the studio. When the button is clicked the
listener is asked for permission to use the mic and then on success is
connected to the signaling server.
- Signaling server sends an offer to connect to the webcaster host,
which then (after the usual WebRTC negotiation) results in a RTC peer
connection between the show and the caller.
- At that point I need to somehow connect the caller's audio stream
(object) into the overall webcaster audio mix so that the conversation
goes out to liquidsoap as usual.
The question I have is where and how within the webcaster client.js do I
connect the callers stream to the outbound audio stream? I'm not an
expert with backbone and underscore (used with webcaster) but I can wing
it as I learn if someone can please point me to the right things to do.
Thanks for you time in considering this!
Mark
------------------------------------------------------------
------------------
Post by Mark E
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
<http://h5ks.mjt.lu/lnk/AEAAHpZTiiEAAAAAAAAAAGgH3uUAASJ-8QAAAAAAAAgArwBYqLzevEDLj7I6TJaVrbRJp9rrAgAHnU0/4/zyopXvsqMbkD2oVwtJYtrg/aHR0cDovL3NkbS5saW5rL3NsYXNoZG90>
Post by Mark E
_______________________________________________
Savonet-users mailing list
https://lists.sourceforge.net/lists/listinfo/savonet-users
<http://h5ks.mjt.lu/lnk/AEAAHpZTiiEAAAAAAAAAAGgH3uUAASJ-8QAAAAAAAAgArwBYqLzevEDLj7I6TJaVrbRJp9rrAgAHnU0/5/lrI1j56e2CZ5Q_oY4_Fzjg/aHR0cHM6Ly9saXN0cy5zb3VyY2Vmb3JnZS5uZXQvbGlzdHMvbGlzdGluZm8vc2F2b25ldC11c2Vycw>
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
<http://h5ks.mjt.lu/lnk/AEAAHpZTiiEAAAAAAAAAAGgH3uUAASJ-8QAAAAAAAAgArwBYqLzevEDLj7I6TJaVrbRJp9rrAgAHnU0/6/I6B7s1SSgVJsEYseisVBlA/aHR0cDovL3NkbS5saW5rL3NsYXNoZG90>
_______________________________________________
Savonet-users mailing list
https://lists.sourceforge.net/lists/listinfo/savonet-users
<http://h5ks.mjt.lu/lnk/AEAAHpZTiiEAAAAAAAAAAGgH3uUAASJ-8QAAAAAAAAgArwBYqLzevEDLj7I6TJaVrbRJp9rrAgAHnU0/7/DXyNUZPMbQOtbSZvrp6xig/aHR0cHM6Ly9saXN0cy5zb3VyY2Vmb3JnZS5uZXQvbGlzdHMvbGlzdGluZm8vc2F2b25ldC11c2Vycw>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot <http://h5ks.mjt.lu/lnk/AEAAHpZTiiEAAAAAAAAAAGgH3uUAASJ-8QAAAAAAAAgArwBYqLzevEDLj7I6TJaVrbRJp9rrAgAHnU0/8/K2ycuW5EtWk5NBJGv9pQIA/aHR0cDovL3NkbS5saW5rL3NsYXNoZG90>
_______________________________________________
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Savonet-users mailing list
https://lists.sourceforge.net/lists/listinfo/savonet-users
Loading...