Investigate Jitsi Meet and alternatives #148

New Issue

raucao · 2020-03-27T20:09:53Z

raucao commented

2020-03-27 20:09:53 +00:00

As it is based on XMPP, I think it only needs the videobridge and one other program. No idea, hence this issue.

(Someone offered to donate a considerable amount, in case we offer it for their Kosmos account.)

As it is based on XMPP, I think it only needs the videobridge and one other program. No idea, hence this issue. (Someone offered to donate a considerable amount, in case we offer it for their Kosmos account.)

raucao self-assigned this 2020-03-27 20:09:53 +00:00

raucao commented

2020-03-27 22:23:38 +00:00

Found the quick install guide and it seems like we really just need to install an apt package and set up an nginx vhost.

I'll give it a try.

Found the [quick install guide](https://github.com/jitsi/jitsi-meet/blob/master/doc/quick-install.md) and it seems like we really just need to install an apt package and set up an nginx vhost. I'll give it a try.

raucao commented

2020-03-27 22:24:31 +00:00

Notes (translate to Chef recipe):

Add meet domain to loopback host entry:

 sudo sed -i 's/^127.0.1.1.*$/127.0.1.1 meet.kosmos.chat meet/g' /etc/hosts

Open firewall for meet traffic:
```
 sudo ufw allow in 10000:20000/udp
```
Add nginx vhost with LE cert (find example config and adapt)
Add universe apt repo (required for deps):
```
 apt-add-repository universe
```

Add the package repo

 echo 'deb https://download.jitsi.org stable/' >> /etc/apt/sources.list.d/jitsi-stable.list
 wget -qO -  https://download.jitsi.org/jitsi-key.gpg.key | apt-key add -

Install package, but with flag that prevents adding nginx vhost (conflicts with existing 443 vhosts)
```
 apt install --no-install-recommends jitsi-meet
```
Increase systemd limits for processes and open files. In /etc/systemd/system.conf:
```
 DefaultLimitNOFILE=65000
 DefaultLimitNPROC=65000
 DefaultTasksMax=65000
```

Notes (translate to Chef recipe): 1. Add meet domain to loopback host entry: sudo sed -i 's/^127.0.1.1.*$/127.0.1.1 meet.kosmos.chat meet/g' /etc/hosts 1. Open firewall for meet traffic: sudo ufw allow in 10000:20000/udp 1. Add nginx vhost with LE cert (find example config and adapt) 1. Add `universe` apt repo (required for deps): apt-add-repository universe 1. Add the package repo echo 'deb https://download.jitsi.org stable/' >> /etc/apt/sources.list.d/jitsi-stable.list wget -qO - https://download.jitsi.org/jitsi-key.gpg.key | apt-key add - 1. Install package, but with flag that prevents adding nginx vhost (conflicts with existing 443 vhosts) apt install --no-install-recommends jitsi-meet 1. Increase systemd limits for processes and open files. In `/etc/systemd/system.conf`: DefaultLimitNOFILE=65000 DefaultLimitNPROC=65000 DefaultTasksMax=65000

raucao commented

2020-03-28 18:29:38 +00:00

Also good to know: performance of the videobridge seems to be pretty good. When they introduced it last year, they said it would be orders of magnitude better than what they had before.

https://jitsi.org/jitsi-videobridge-performance-evaluation/

On a plain Xeon server [...] for about 20% CPU you will be able to run 1000+ video streams using an average of 550 Mbps!

We performed all tests on an unmodified Jitsi Meet installation, on a dedicated server. The server is running all the services needed by Jitsi Meet: a web-server (nginx), an XMPP server (prosody) and Jitsi Videobridge. The server machine is equipped with a quad-core Intel ® Xeon® E5-1620 v2 @ 3.70GHz CPU.

Also good to know: performance of the videobridge seems to be pretty good. When they introduced it last year, they said it would be orders of magnitude better than what they had before. https://jitsi.org/jitsi-videobridge-performance-evaluation/ > On a plain Xeon server [...] for about 20% CPU you will be able to run 1000+ video streams using an average of 550 Mbps! > We performed all tests on an unmodified Jitsi Meet installation, on a dedicated server. The server is running all the services needed by Jitsi Meet: a web-server (nginx), an XMPP server (prosody) and Jitsi Videobridge. The server machine is equipped with a quad-core Intel ® Xeon® E5-1620 v2 @ 3.70GHz CPU.

raucao commented

2020-03-29 02:42:34 +00:00

I checked again, and Andromeda is actually not too far from RAM limits, and also using considerable CPU resources already. So it would probably make more sense to throw Jitsi on the new Centaurus, which is going to be online soon.

I checked again, and Andromeda is actually not too far from RAM limits, and also using considerable CPU resources already. So it would probably make more sense to throw Jitsi on the new Centaurus, which [is going to be online soon](https://gitea.kosmos.org/kosmos/chef/issues/147#issuecomment-1774).

raucao commented

2020-03-30 20:00:17 +00:00

From my first quick try, it looks like Jitsi Meet is requires Prosody as XMPP server. :/

Have to investigate further. The jisti-meet package installs many different packages (incl. Prosody), which, in combination, make up the system.

From my first quick try, it looks like Jitsi Meet is requires Prosody as XMPP server. :/ Have to investigate further. The `jisti-meet` package installs many different packages (incl. Prosody), which, in combination, make up the system.

raucao commented

2020-04-21 08:19:50 +00:00

Just got linked to exactly what we need, by @stevenroose@x0f.org:

https://blog.jabberhead.tk/2020/03/16/install-jitsi-meet-alongside-ejabberd/

Just got linked to exactly what we need, by @stevenroose@x0f.org: https://blog.jabberhead.tk/2020/03/16/install-jitsi-meet-alongside-ejabberd/

raucao commented

2020-05-07 08:08:10 +00:00

Just found out it's possible to set up normal XMPP auth for Jitsi Meet room creation after all:

https://www.howtoforge.com/how-to-install-jitsi-meet-video-conferencing-solution-on-debian-10/#set-up-user-authentication-for-jitsi-meet

Just found out it's possible to set up normal XMPP auth for Jitsi Meet room creation after all: https://www.howtoforge.com/how-to-install-jitsi-meet-video-conferencing-solution-on-debian-10/#set-up-user-authentication-for-jitsi-meet

raucao commented

2020-09-23 11:18:33 +00:00

This is becoming a priority with a deadline now, so we can use an opensource tool for remote participants at the upcoming Kosmos Summit.

Ideally using existing Kosmos XMPP/LDAP accounts for permission to create rooms, of course. And probably even kosmos.chat MUC rooms for the chat in Jitsi.

This is becoming a priority with a deadline now, so we can use an opensource tool for remote participants at the upcoming [Kosmos Summit](https://wiki.kosmos.org/Kosmos_Summit_2020). Ideally using existing Kosmos XMPP/LDAP accounts for permission to create rooms, of course. And probably even kosmos.chat MUC rooms for the chat in Jitsi.

raucao commented

2020-09-24 11:30:07 +00:00

If we want to record meetings/sessions, there's a Jitsi module called Jibri:

Jibri provides services for recording or streaming a Jitsi Meet conference.

It works by launching a Chrome instance rendered in a virtual framebuffer and capturing and encoding the output with ffmpeg. It is intended to be run on a separate machine (or a VM), with no other applications using the display or audio devices. Only one recording at a time is supported on a single jibri.

https://github.com/jitsi/jibri

If we want to record meetings/sessions, there's a Jitsi module called Jibri: > Jibri provides services for recording or streaming a Jitsi Meet conference. > > It works by launching a Chrome instance rendered in a virtual framebuffer and capturing and encoding the output with ffmpeg. It is intended to be run on a separate machine (or a VM), with no other applications using the display or audio devices. Only one recording at a time is supported on a single jibri. https://github.com/jitsi/jibri

raucao commented

2020-09-24 11:34:46 +00:00

I looked into BigBlueButton this morning, because someone recommended it to me on fedi, after I asked about remote confs with whiteboards.

Summary of my findings, copied over from chat:

seems OK, but not sure it's better than jitsi meet
it doesn't come with any user accounts by default, but just an API
but there's a rails app for managing meetings and recordings, and it supports LDAP
so that could work
it's similar to zoom i guess, api-wise
but for kosmos chat and as a service, it would obviously be cool to just use plain xmpp for meeting management and permissions
much easier to do from hyperchannel then, because a user doesn't have direct API access
bigbluebutton shows a collaborative whiteboard by default, which is why it was recommended to me
so, to sum it up: i guess the easy and automatable recordings are the biggest benefit vs jitsi meet
and the whiteboard would handily solve that problem on the side
but not sure it's great as an overall kosmos service

I looked into [BigBlueButton](https://docs.bigbluebutton.org) this morning, because someone recommended it to me on fedi, after I asked about remote confs with whiteboards. Summary of my findings, copied over from chat: > seems OK, but not sure it's better than jitsi meet > it doesn't come with any user accounts by default, but just an API > but there's a rails app for managing meetings and recordings, and it supports LDAP > so that could work > it's similar to zoom i guess, api-wise > but for kosmos chat and as a service, it would obviously be cool to just use plain xmpp for meeting management and permissions > much easier to do from hyperchannel then, because a user doesn't have direct API access > bigbluebutton shows a collaborative whiteboard by default, which is why it was recommended to me > so, to sum it up: i guess the easy and automatable recordings are the biggest benefit vs jitsi meet > and the whiteboard would handily solve that problem on the side > but not sure it's great as an overall kosmos service

raucao commented

2020-10-22 09:20:33 +00:00

I guess BigBlueButton is out of the question... https://www.redteam-pentesting.de/en/advisories/rt-sa-2020-005/-arbitrary-file-disclosure-and-server-side-request-forgery-in-bigbluebutton

raucao commented

2022-10-06 08:32:07 +00:00

For reference:

I'm thinking:

Add jitsi nginx vhost to our nginx proxy role
Prepare ejabberd to be used with Jitsi (i.e. configure new MUC domain for meeting rooms)
Create a VM with Jitsi Meet and jicofo, route traffic from the nginx vhost to it
Create one VM per host (or just one to begin with), with jitsi-videobridge, route UDP traffic on port 10000 to those

Option 2: We could also just skip using our own ejabberd, but then we'd need to add authenticated room creation functionality to some other Web front-end of ours. I guess accounts.kosmos.org would be the natural choice. Not sure if that's better because we control the UX, or worse because it's not directly integrated in the Jitsi Meet front-end.

For reference: * https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-scalable * https://blog.mi.hdm-stuttgart.de/index.php/2021/03/11/how-to-scale-jitsi-meet/ I'm thinking: * Add jitsi nginx vhost to our nginx proxy role * Prepare ejabberd to be used with Jitsi (i.e. configure new MUC domain for meeting rooms) * Create a VM with Jitsi Meet and jicofo, route traffic from the nginx vhost to it * Create one VM per host (or just one to begin with), with jitsi-videobridge, route UDP traffic on port 10000 to those Option 2: We could also just skip using our own ejabberd, but then we'd need to add authenticated room creation functionality to some other Web front-end of ours. I guess `accounts.kosmos.org` would be the natural choice. Not sure if that's better because we control the UX, or worse because it's not directly integrated in the Jitsi Meet front-end.

raucao commented

2023-04-11 10:52:40 +00:00

It's probably a blessing that we haven't deployed Jitsi in production yet, because the OpenTalk source code is now available, and they have a Docker Compose setup, too:

https://gitlab.opencode.de/opentalk/ot-setup

I think the whole architecture lends itself much better to integrate with our setup. Here are the services that the official "Lite" setup provides, plus notes for Kosmos infra/deployment:

Service	core	Kosmos
Keycloak	X	Akkounts, or Keycloak with our LDAP
autoheal	X
rabbitmq	X	ejabberd if MQTT, or new RabbitMQ instance, or local
redis	X	Redis (same as RS), or local
web-frontend	X	nginx proxy (static app)
controller	X
minio	X	Garage
janus-gateway	X
obelisk
smtp-mailer
spacedeck
etherpad

Here's an overview of the architecture, screenshot from a recent talk (in German):

Note: Kubernetes is entirely optional. However, the system is designed to spawn a new container for every conference, which is nice because if you don't persist anything intentionally (like recordings to S3), then all logs and data are torn down with the container when the meeting/conference ends. Also, it means one instance or even the controller crashing or stalling doesn't impact other running conferences.

It's probably a blessing that we haven't deployed Jitsi in production yet, because the OpenTalk [source code is now available](https://gitlab.opencode.de/opentalk), and they have a Docker Compose setup, too: https://gitlab.opencode.de/opentalk/ot-setup I think the whole architecture lends itself much better to integrate with our setup. Here are the services that the official "Lite" setup provides, plus notes for Kosmos infra/deployment: | Service | core | Kosmos | |--------------|-----------|-----------| | Keycloak | X | Akkounts, or Keycloak with our LDAP | | autoheal | X | | | rabbitmq | X | ejabberd if MQTT, or new RabbitMQ instance, or local | | redis | X | Redis (same as RS), or local | | web-frontend | X | nginx proxy (static app) | | controller | X | | | minio | X | Garage | | janus-gateway| X | | | obelisk | | | | smtp-mailer | | | | spacedeck | | | | etherpad | | | Here's an overview of the architecture, screenshot from a [recent talk (in German)](https://media.ccc.de/v/clt23-271-digital-souverane-videokonferenzen-in-thuringen): ![Screenshot](https://gitea.kosmos.org/attachments/72972b09-9ddd-42fc-837f-0a38a5cc05c9) Note: Kubernetes is entirely optional. However, the system is designed to spawn a new container for every conference, which is nice because if you don't persist anything intentionally (like recordings to S3), then all logs and data are torn down with the container when the meeting/conference ends. Also, it means one instance or even the controller crashing or stalling doesn't impact other running conferences.

Screenshot from 2023-04-10 14-35-46.png

313 KiB

raucao changed title from ~~Investigate Jitsi Meet requirements~~ to Investigate Jitsi Meet (and alternatives)

2023-04-11 11:00:40 +00:00

raucao changed title from ~~Investigate Jitsi Meet (and alternatives)~~ to Investigate Jitsi Meet and alternatives

2023-04-11 11:00:49 +00:00

Sign in to join this conversation.

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: kosmos/chef#148