Set up LDAP server for central account management #107

Closed
opened 2019-10-15 13:50:15 +00:00 by raucao · 28 comments
Owner

In order to manage accounts for Kosmos services only once, and have all the services use the same account data/passwords/etc., we're considering to use LDAP, which is supported by all of Mastodon, ejabberd, and Gitea.

In order to manage accounts for Kosmos services only once, and have all the services use the same account data/passwords/etc., we're considering to use LDAP, which is supported by all of Mastodon, ejabberd, and Gitea.
greg was assigned by raucao 2019-10-15 13:50:15 +00:00
raucao self-assigned this 2019-10-15 13:50:15 +00:00
Author
Owner
Greg and I looked at a bunch of options, and we're now testing [389 Directory Server](https://directory.fedoraproject.org). Notes: * [LDAP Account Manager (Web front-end)](https://www.ldap-account-manager.org) * [LDAP npm module (for akkounts)](http://ldapjs.org) * [Django self-service account management app (contains code for setting PW)](https://github.com/KENET-KE/django-ldap-user-registration) * [Nostalgia galore](http://www.yolinux.com/TUTORIALS/LinuxTutorialaWebDap.html) * [389 Chef cookbook](https://supermarket.chef.io/cookbooks/dirsrv) * [389 systemd howto](https://directory.fedoraproject.org/docs/389ds/howto/howto-systemd.html) * [389 development roadmap](https://directory.fedoraproject.org/docs/389ds/FAQ/roadmap.html) * [https://www.digitalocean.com/community/tutorials/understanding-the-ldap-protocol-data-hierarchy-and-entry-components](https://www.digitalocean.com/community/tutorials/understanding-the-ldap-protocol-data-hierarchy-and-entry-components)
Author
Owner

This contains interesting information, even when not using GitLab: https://docs.gitlab.com/ee/administration/auth/ldap.html

This contains interesting information, even when not using GitLab: https://docs.gitlab.com/ee/administration/auth/ldap.html
Owner
[Docs about LDAP configuration in ejabberd](https://docs.ejabberd.im/admin/configuration/#ldap)
Author
Owner

So a decent integration always supports a custom "filter" config, which can be filled with the desired attributes, in order to select users that are allowed to use that service. Saw that in the GitLab doc as well.

So a decent integration always supports a custom "filter" config, which can be filled with the desired attributes, in order to select users that are allowed to use that service. Saw that in the GitLab doc as well.
Owner

I got ejabberd working with LDAP auth in a VM. ejabberd itself, like GitLab, only has read-only support for LDAP, so you can't create users on LDAP through ejabberdctl. I created an ldif file and used ldapadd to add a group and a user:

host_config:
  "kosmos.org":
    [snip]
    auth_method: [ldap]
    ldap_servers: ["vagrant.vm"]
    ## ldap_rootdn: "cn=Directory Manager,dc=kosmos,dc=org"
    ldap_rootdn: "cn=Directory Manager"
    ldap_password: "ldap_password"
    ## ldap_encrypt: tls
    ldap_port: 389
    ## Define the user's base
    ldap_base: "ou=users,dc=kosmos,dc=org"
    ldap_filter: "(objectClass=account)"

My ldif file, imported using ldapadd -x -W -D "cn=Directory Manager" -f users.ldif:

# users.ldif
dn: cn=xmpp_users,dc=kosmos,dc=org
cn: xmpp_users
objectclass: groupofNames
member: cn=greg,ou=users,dc=kosmos,dc=org

dn: cn=greg,ou=users,dc=kosmos,dc=org
objectClass: top
objectClass: account
objectClass: person
cn: greg
sn: greg
uid: greg
userPassword: thisisatest

Interesting for the migration, ejabberd allows you to enable multiple authentication methods, so we can still keep sql for current users but add ldap:

auth_method: [sql, ldap]

So far I have not figured out how to filter out users belonging to a group, so for now in this config I'm considering every account has access to XMPP. I have found out that 389 does not have built-in support for memberOf, which is a way to filter users by groups, however there is a plugin for it (https://directory.fedoraproject.org/docs/389ds/design/memberof-plugin.html) and probably other ways to filter rather than groups, maybe attributes are better for this

I got ejabberd working with LDAP auth in a VM. ejabberd itself, like GitLab, only has read-only support for LDAP, so you can't create users on LDAP through ejabberdctl. I created an ldif file and used ldapadd to add a group and a user: ```yml host_config: "kosmos.org": [snip] auth_method: [ldap] ldap_servers: ["vagrant.vm"] ## ldap_rootdn: "cn=Directory Manager,dc=kosmos,dc=org" ldap_rootdn: "cn=Directory Manager" ldap_password: "ldap_password" ## ldap_encrypt: tls ldap_port: 389 ## Define the user's base ldap_base: "ou=users,dc=kosmos,dc=org" ldap_filter: "(objectClass=account)" ``` My ldif file, imported using `ldapadd -x -W -D "cn=Directory Manager" -f users.ldif`: ```ldif # users.ldif dn: cn=xmpp_users,dc=kosmos,dc=org cn: xmpp_users objectclass: groupofNames member: cn=greg,ou=users,dc=kosmos,dc=org dn: cn=greg,ou=users,dc=kosmos,dc=org objectClass: top objectClass: account objectClass: person cn: greg sn: greg uid: greg userPassword: thisisatest ``` Interesting for the migration, ejabberd allows you to enable multiple authentication methods, so we can still keep sql for current users but add ldap: ``` auth_method: [sql, ldap] ``` So far I have not figured out how to filter out users belonging to a group, so for now in this config I'm considering every `account` has access to XMPP. I have found out that 389 does not have built-in support for memberOf, which is a way to filter users by groups, however there is a plugin for it (https://directory.fedoraproject.org/docs/389ds/design/memberof-plugin.html) and probably other ways to filter rather than groups, maybe attributes are better for this
Owner

On the subject of groups: https://ldapwiki.com/wiki/Groups%20Are%20Bad

RBAC could be what we want

On the subject of groups: https://ldapwiki.com/wiki/Groups%20Are%20Bad [RBAC](https://ldapwiki.com/wiki/RBAC) could be what we want
Author
Owner

Not sure what you mean. We can just use attributes to know if a user has access to a certain service or not.

Not sure what you mean. We can just use attributes to know if a user has access to a certain service or not.
Owner

Got it, using extensibleObject as objectClass we can used arbitrary attributes:

# users.ldif
dn: cn=greg,ou=users,dc=kosmos,dc=org
objectClass: top
objectClass: account
objectClass: person
objectClass: extensibleObject
cn: greg
sn: greg
uid: greg
xmpp: enabled
userPassword: lalalala

Then we can use the attribute as a filter in ejabberd.yml:

ldap_filter: "(&(objectClass=account)(xmpp=enabled))"
Got it, using `extensibleObject` as `objectClass` we can used arbitrary attributes: ```ldif # users.ldif dn: cn=greg,ou=users,dc=kosmos,dc=org objectClass: top objectClass: account objectClass: person objectClass: extensibleObject cn: greg sn: greg uid: greg xmpp: enabled userPassword: lalalala ``` Then we can use the attribute as a filter in ejabberd.yml: ```yml ldap_filter: "(&(objectClass=account)(xmpp=enabled))" ```
Author
Owner

I was thinking more like service=xmpp,service=mastodon and so on. Later, it will also need extra attributes for which namespace of which service it is.

Edit: so actually, it could already be service=xmpp.kosmos.org for example, unless dots aren't allowed in values.

I was thinking more like `service=xmpp,service=mastodon` and so on. Later, it will also need extra attributes for which namespace of which service it is. Edit: so actually, it could already be `service=xmpp.kosmos.org` for example, unless dots aren't allowed in values.
Owner

Good idea, that works.

dn: cn=greg,ou=users,dc=kosmos,dc=org
objectClass: top
objectClass: account
objectClass: person
objectClass: extensibleObject
cn: greg
sn: greg
uid: greg
service: xmpp.kosmos.org
service: kosmos.social
userPassword: lalalala
ldap_filter: "(&(objectClass=account)(service=xmpp.kosmos.org))"
Good idea, that works. ```ldif dn: cn=greg,ou=users,dc=kosmos,dc=org objectClass: top objectClass: account objectClass: person objectClass: extensibleObject cn: greg sn: greg uid: greg service: xmpp.kosmos.org service: kosmos.social userPassword: lalalala ``` ```yml ldap_filter: "(&(objectClass=account)(service=xmpp.kosmos.org))" ```
Owner

I got Mastodon to work with LDAP in a VM.

Logging in through the Mastodon web interface with an LDAP user (using the username or email) creates an account in the database. Similar to ejabberd you can set the filter you want to select the accounts. Right now my config is set to:

LDAP_ENABLED=true
LDAP_HOST="localhost"
LDAP_PORT=389
LDAP_METHOD=:simple #disable TLS
LDAP_BASE="ou=users,dc=kosmos,dc=org"
LDAP_BIND_DN="cn=Directory Manager"
LDAP_PASSWORD="password"
LDAP_UID="uid"
LDAP_SEARCH_FILTER="(&(objectClass=account)(service=kosmos.social)(|(%{uid}=%{email})(mail=%{email})))"

With my current settings I can log in with a username and password, but not with an email

Edit: Updated the config, now logging in is possible with the username or email

I have added an email field to my test user in the ldif file because Mastodon needs an email.

dn: cn=greg,ou=users,dc=kosmos,dc=org
objectClass: top
objectClass: account
objectClass: person
objectClass: extensibleObject
cn: greg
sn: greg
uid: greg
mail: greg@kosmos.org
service: xmpp.kosmos.org
service: kosmos.social
userPassword: lalalala
I got Mastodon to work with LDAP in a VM. Logging in through the Mastodon web interface with an LDAP user (using the username or email) creates an account in the database. Similar to ejabberd you can set the filter you want to select the accounts. Right now my config is set to: ``` LDAP_ENABLED=true LDAP_HOST="localhost" LDAP_PORT=389 LDAP_METHOD=:simple #disable TLS LDAP_BASE="ou=users,dc=kosmos,dc=org" LDAP_BIND_DN="cn=Directory Manager" LDAP_PASSWORD="password" LDAP_UID="uid" LDAP_SEARCH_FILTER="(&(objectClass=account)(service=kosmos.social)(|(%{uid}=%{email})(mail=%{email})))" ``` <strike>With my current settings I can log in with a username and password, but not with an email</strike> Edit: Updated the config, now logging in is possible with the username or email I have added an email field to my test user in the ldif file because Mastodon needs an email. ```ldif dn: cn=greg,ou=users,dc=kosmos,dc=org objectClass: top objectClass: account objectClass: person objectClass: extensibleObject cn: greg sn: greg uid: greg mail: greg@kosmos.org service: xmpp.kosmos.org service: kosmos.social userPassword: lalalala ```
Author
Owner

Great!

Does that mean we can actually use the existing accounts? Or that we can at least migrate the existing accounts easily by adding those users to the LDAP directory?

Great! Does that mean we can actually use the existing accounts? Or that we can at least migrate the existing accounts easily by adding those users to the LDAP directory?
Owner

Existing accounts will work after we enable LDAP in Mastodon. Accounts created by logging in with an LDAP user do not have an encrypted_password set in the Mastodon database, the password is checked on the LDAP account and they have the external flag set. Existing accounts that have an encrypted_password set will still work if they do not exist in LDAP.

On log in with an account and password from LDAP it will find or create the user: a1f04c1e34/app/models/concerns/ldap_authenticable.rb (L16)

The way it is done right now in Mastodon, logins will not work when LDAP is enabled and the LDAP server is down, even if your account has an encrypted_password set (exception even if the user is not from LDAP)

Existing accounts will work after we enable LDAP in Mastodon. Accounts created by logging in with an LDAP user do not have an `encrypted_password` set in the Mastodon database, the password is checked on the LDAP account and they have the `external` flag set. Existing accounts that have an `encrypted_password` set will still work if they do not exist in LDAP. On log in with an account and password from LDAP it will find or create the user: https://github.com/tootsuite/mastodon/blob/a1f04c1e3497e9dff5970038461d9f454f2650df/app/models/concerns/ldap_authenticable.rb#L16 The way it is done right now in Mastodon, logins will not work when LDAP is enabled and the LDAP server is down, even if your account has an `encrypted_password` set (exception even if the user is not from LDAP)
Author
Owner

On log in with an account and password from LDAP it will find or create the user

I'm pretty sure we do not want to create users from Mastodon. It would mean that when you already have an LDAP user for e.g. XMPP or Gitea, you then have an additional one, no?

The way it is done right now in Mastodon, logins will not work when LDAP is enabled and the LDAP server is down, even if your account has an encrypted_password set (exception even if the user is not from LDAP)

Yes, of course. Everything else would be weird in my opinion. That's why you can usually have multiple LDAP masters and clients can try an alternative one, if the first one is down (afaics).

> On log in with an account and password from LDAP it will find or create the user I'm pretty sure we do not want to create users from Mastodon. It would mean that when you already have an LDAP user for e.g. XMPP or Gitea, you then have an additional one, no? > The way it is done right now in Mastodon, logins will not work when LDAP is enabled and the LDAP server is down, even if your account has an encrypted_password set (exception even if the user is not from LDAP) Yes, of course. Everything else would be weird in my opinion. That's why you can usually have multiple LDAP masters and clients can try an alternative one, if the first one is down (afaics).
Owner

I’m pretty sure we do not want to create users from Mastodon. It would mean that when you already have an LDAP user for e.g. XMPP or Gitea, you then have an additional one, no?

I don't understand the question. Mastodon will always create a user in the database after a successful login via LDAP, if the user isn't in the database already. This user is set as external, and has no password, since the one from LDAP is used

> I’m pretty sure we do not want to create users from Mastodon. It would mean that when you already have an LDAP user for e.g. XMPP or Gitea, you then have an additional one, no? I don't understand the question. Mastodon will always create a user in the database after a successful login via LDAP, if the user isn't in the database already. This user is set as external, and has no password, since the one from LDAP is used
Author
Owner

If that's the case, then how are accounts the same between Mastodon and other services?

If that's the case, then how are accounts the same between Mastodon and other services?
Owner

If that’s the case, then how are accounts the same between Mastodon and other services?

Think of it like a OAuth flow. On successful auth with LDAP from Mastodon, a User object is persisted in the Mastodon database if it does not already exist, and the user is logged in

> If that’s the case, then how are accounts the same between Mastodon and other services? Think of it like a OAuth flow. On successful auth with LDAP from Mastodon, a User object is persisted in the Mastodon database if it does not already exist, and the user is logged in
Author
Owner

I think you still don't understand my question. In any case, the point is that Mastodon should never create new accounts, because those accounts then have no information about our other services stored.

I think you still don't understand my question. In any case, the point is that Mastodon should never create new accounts, because those accounts then have no information about our other services stored.
Owner

That's correct, I still don't understand your question.

Let's start from the beginning. Here is how things would work:

A user donates using akkounts to create an account. After we receive the transaction they get to pick a username and password, we create an LDAP account. Then they can use this username and password to log into Mastodon. Mastodon sees the LDAP account exists, creates an entry for the user in the Mastodon database, with an external flag set to true, and no encrypted_password since the password lives in LDAP. With these same credentials you can log into our wiki and XMPP server too

Mastodon should never create new accounts, because those accounts then have no information about our other services stored.

What do you mean by Mastodon creating new accounts? Are you talking about LDAP accounts being created by Mastodon somehow? This is definitely not something that makes any sense

By the way I just got Mediawiki to work with LDAP login in my VM, using https://www.mediawiki.org/wiki/Extension:LDAPAuthentication2

That's correct, I still don't understand your question. Let's start from the beginning. Here is how things would work: A user donates using akkounts to create an account. After we receive the transaction they get to pick a username and password, we create an LDAP account. Then they can use this username and password to log into Mastodon. Mastodon sees the LDAP account exists, creates an entry for the user in the Mastodon database, with an `external` flag set to true, and no `encrypted_password` since the password lives in LDAP. With these same credentials you can log into our wiki and XMPP server too > Mastodon should never create new accounts, because those accounts then have no information about our other services stored. What do you mean by Mastodon creating new accounts? Are you talking about LDAP accounts being created by Mastodon somehow? This is definitely not something that makes any sense By the way I just got Mediawiki to work with LDAP login in my VM, using https://www.mediawiki.org/wiki/Extension:LDAPAuthentication2
Author
Owner

What do you mean by Mastodon creating new accounts? Are you talking about LDAP accounts being created by Mastodon somehow? This is definitely not something that makes any sense

This is exactly how I read what you wrote before. So we're on the same page about it now.

> What do you mean by Mastodon creating new accounts? Are you talking about LDAP accounts being created by Mastodon somehow? This is definitely not something that makes any sense This is exactly how I read what you wrote before. So we're on the same page about it now.
Author
Owner

By the way I just got Mediawiki to work with LDAP login in my VM, using https://www.mediawiki.org/wiki/Extension:LDAPAuthentication2

That's great! It'll be much less friction not having to sign up for a wiki account, and every user should have one by default in my opinion.

> By the way I just got Mediawiki to work with LDAP login in my VM, using https://www.mediawiki.org/wiki/Extension:LDAPAuthentication2 That's great! It'll be much less friction not having to sign up for a wiki account, and every user should have one by default in my opinion.
Owner

Making the TLS setup took me longer than I thought (it involved some funky format changes), but I got it to work. I'm going to push a PR later this evening or tomorrow

Making the TLS setup took me longer than I thought (it involved some funky format changes), but I got it to work. I'm going to push a PR later this evening or tomorrow
Owner

Here's a snippet to create a hashed + salted SHA512 password that can be used as a userPassword:

$ irb -r base64 -r digest -r securerandom
>> password =  "yourpassword"
>> salt = SecureRandom.hex(32)
>> hash = "{SSHA512}" + Base64.strict_encode64(Digest::SHA512.digest(password+salt) + salt)
=> "{SSHA512}mpCXkUuSxETLcuqJRuZxMeU8rESHhLhDOP1r/9TKKPVv/DxupY8sQg5WR1k1dtzUhL7DVq2VpL+X94+AmsV0pzgxNWQ4MTdhYTFkNjMxZGYwYTA3YTNiOGQ3ZTEzOTZkOTE2YWZhMDUyZGY4ZWJmMWYxZTgzMTE0MjdiZDNiY2M="

The encrypted passwords in the Mastodon database are using bcrypt, and 389-ds does not support it, only MD5, SHA and PBKDF2 (https://pagure.io/389-ds-base/issue/397). To migrate the existing kosmos.social user accounts I think we need to write a Mastodon extension:

We would create a user document in LDAP for every Mastodon user with their username and email, setting their password to a long random string. They can still log in using their existing password. The extension would ask them for a new password, that would be on their LDAP user document, and their User in the Mastodon database would have the encrypted_password attribute set to "", and they can now log into their Mastodon account using LDAP with the new password they have set, as well as Mediawiki and XMPP

Without making it a Mastodon extension I'm not sure how users could prove they own the username/email. An email would look super fishy/phishy. Does someone have a better idea?

Here's a snippet to create a hashed + salted SHA512 password that can be used as a userPassword: ``` $ irb -r base64 -r digest -r securerandom >> password = "yourpassword" >> salt = SecureRandom.hex(32) >> hash = "{SSHA512}" + Base64.strict_encode64(Digest::SHA512.digest(password+salt) + salt) => "{SSHA512}mpCXkUuSxETLcuqJRuZxMeU8rESHhLhDOP1r/9TKKPVv/DxupY8sQg5WR1k1dtzUhL7DVq2VpL+X94+AmsV0pzgxNWQ4MTdhYTFkNjMxZGYwYTA3YTNiOGQ3ZTEzOTZkOTE2YWZhMDUyZGY4ZWJmMWYxZTgzMTE0MjdiZDNiY2M=" ``` The encrypted passwords in the Mastodon database are using bcrypt, and 389-ds does not support it, only MD5, SHA and PBKDF2 (https://pagure.io/389-ds-base/issue/397). To migrate the existing kosmos.social user accounts I think we need to write a Mastodon extension: We would create a user document in LDAP for every Mastodon user with their username and email, setting their password to a long random string. They can still log in using their existing password. The extension would ask them for a new password, that would be on their LDAP user document, and their `User` in the Mastodon database would have the `encrypted_password` attribute set to `""`, and they can now log into their Mastodon account using LDAP with the new password they have set, as well as Mediawiki and XMPP Without making it a Mastodon extension I'm not sure how users could prove they own the username/email. An email would look super fishy/phishy. Does someone have a better idea?
Owner

I think this one can be closed since we merged #115 and created a new issue for ejabberd support. I will create other issues when needed

I think this one can be closed since we merged #115 and created a new issue for ejabberd support. I will create other issues when needed
greg closed this issue 2020-01-29 15:39:50 +00:00
Author
Owner

Re-opening until the LDAP server is documented. As of now, nobody but us would be able to understand what happened.

Re-opening until the LDAP server is documented. As of now, nobody but us would be able to understand what happened.
raucao reopened this issue 2020-01-29 15:43:47 +00:00
Owner

We've been discussing ways to support multiple domains in the ops room on XMPP, to support other domains for Kosmos services.

raucao found this post that's interesting and proposes 3 ways of doing it: https://serverfault.com/questions/828490/setting-up-multiple-domain-in-ldap-server#828497

Variant 2 seems like a good way to go. I think we should be able to migrate our current users to this hierarchy before doing anything else

This is what it would look like:

dc=kosmos,dc=org
    cn=users
        ou=kosmos.org
            uid=alice
            uid=bob
        ou=customer.pro 
            uid=claire
We've been discussing ways to support multiple domains in the ops room on XMPP, to support other domains for Kosmos services. raucao found this post that's interesting and proposes 3 ways of doing it: https://serverfault.com/questions/828490/setting-up-multiple-domain-in-ldap-server#828497 Variant 2 seems like a good way to go. I think we should be able to migrate our current users to this hierarchy before doing anything else This is what it would look like: ``` dc=kosmos,dc=org cn=users ou=kosmos.org uid=alice uid=bob ou=customer.pro uid=claire ```
Owner

We also need to plan groups as part of the hierarchy, for example for Gitea orgs

We also need to plan groups as part of the hierarchy, for example for Gitea orgs
Owner

Docs are on the wiki for the new directory structure: https://wiki.kosmos.org/Infrastructure:LDAP

Docs are on the wiki for the new directory structure: https://wiki.kosmos.org/Infrastructure:LDAP
greg closed this issue 2020-02-20 13:39:20 +00:00
Sign in to join this conversation.
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: kosmos/chef#107
No description provided.