- [docker](Dockerfile) Update Z-Push package URL.

- [kopano](src/kopano) Now use ubuntu:20.04 in `kopano-webaddr.sh`.

CHANGELOG.md

@@ -1,3 +1,54 @@
+# 1.3.1
+
+- [docker](Dockerfile) Update Z-Push package URL.
+- [kopano](src/kopano) Now use ubuntu:20.04 in `kopano-webaddr.sh`.
+- [docker](ROADMAP.md) Update roadmap.
+
+# 1.3.0
+
+- [docker](Dockerfile) Now use ubuntu:20.04 as base image.
+- [docker](Makefile) Don't set BLD_ARG in Makefile.
+- [kopano](src/kopano/plugin/movetopublicldap.py) Need explicitly to set `allow_no_value=True` due to update in `configparser.3.8.py`.
+- [demo](demo) Update to `mta-apk_list` target.
+
+# 1.2.9
+
+- [kopano](src/kopano) Bug in kopano-dagent out-of-office now fixed.
+- [demo](demo) Make demo less dependent on that curl and firefox beeing installed on the host.
+- [demo](demo) Don't expose any ports on the host, avoiding potential conflicts.
+
+# 1.2.8
+
+- [test](test) Update to use `mlan/openldap:2`.
+- [demo](demo) Update to use `mlan/openldap:2`.
+
+# 1.2.7
+
+- [kopano](src/kopano/plugin/movetopublicldap.py) Add LDAP support to the [Move to public](https://documentation.kopano.io/kopanocore_administrator_manual/special_kc_configurations.html#move-to-public ) kopano-dagent python plugin.
+- [kopano](src/kopano/plugin/movetopublicldap.py) Add sample config file for the kopano-dagent python plugin movetopublicldap.
+
+# 1.2.6
+
+- [docker](src/docker/bin/docker-config.sh) Allow the crontab support to work also when the file `/etc/kopano/docker-crontab` is missing.
+
+# 1.2.5
+
+- [docker](Dockerfile) Install the [kopano archiver](https://documentation.kopano.io/kopano_archiver_manual/) in target `core`.
+- [docker](Dockerfile) Install the smime webapp plugin providing [S/MIME](https://kopano.com/blog/s-mime-plugin-description/).
+- [docker](Dockerfile) Install the mdm webapp plugin providing [Mobile Device Management](https://documentation.kopano.io/webapp_mdm_manual/).
+- [demo](demo) Add [S/MIME](https://kopano.com/blog/s-mime-plugin-description/) cert generation.
+- [kopano](src/kopano/entry.d/10-kopano-common) Fix bug in `kopano_apply_envvars_php()`.
+- [kopano](src/kopano/entry.d/10-kopano-common) Allow module based parameter names.
+- [docker](src/docker/bin/docker-config.sh) Add provision to set up crontab using envvars.
+- [docker](README.md) Updated documentation.
+- [kopano](src/kopano) `MIGRATE_CONFIG=2` make sure WebApp plugins have configuration files in place.
+- [docker](Dockerfile) Make a copy of the kopano-webapp config files.
+
+# 1.2.4
+
+- [kopano](src/kopano) Now, use man pages and sample config files to find valid keys to match envvar.
+- [kopano](src/kopano) Load Kopano LDAP attributes using `LDAP_PROPMAP=`.
+
 # 1.2.3
 
 - [kopano](src/kopano) Adding support for secure IMAPS, POP3S and ICALS.
@@ -74,7 +125,7 @@
 - Added support of the environment variable `LMTP_LISTEN=*:2003`, due to misconfiguration of `kopano-dagent` in recent releases (8.7.84).
 - Simplified the health check.
 - Changed repository directory structure to a more general one.
-- Renamed some build variables, e.g., `DOCKER_RUNSV_DIR` (was `docker_build_runit_root`).
+- Renamed some build variables, e.g., `DOCKER_RUNSV_DIR` (was `docker_build_runit_root`).
 - Cleaning up `Makefile`
 - Added more debug functionality in `demo/Makefile`
 

Dockerfile

@@ -2,7 +2,7 @@
 # build arguments, amd64 is the default
 #
 ARG	DIST=ubuntu
-ARG	REL=18.04
+ARG	REL=20.04
 ARG	ARCH
 
 FROM	${ARCH:+$ARCH/}$DIST:$REL AS base
@@ -13,7 +13,11 @@ ENV	DEBIAN_FRONTEND=noninteractive \
 	DOCKER_BIN_DIR=/usr/local/bin \
 	DOCKER_ENTRY_DIR=/etc/docker/entry.d \
 	DOCKER_EXIT_DIR=/etc/docker/exit.d \
+	DOCKER_CRONTAB_FILE=/etc/kopano/docker-crontab \
+	DOCKER_CRONTAB_DIR=/etc/cron.d \
 	DOCKER_CONF_DIR1=/etc/kopano \
+	DOCKER_SMPL_DIR1=/usr/share/doc/kopano/example-config \
+	DOCKER_PLUG_DIR=/usr/share/kopano-dagent/python/plugins \
 	DOCKER_CONF_DIR2=/usr/share/z-push \
 	DOCKER_APPL_LIB=/var/lib/kopano \
 	DOCKER_APPL_SSL_DIR=/etc/kopano/ssl \
@@ -31,6 +35,8 @@ ENV	DEBIAN_FRONTEND=noninteractive \
 COPY	src/*/bin $DOCKER_BIN_DIR/
 COPY	src/*/entry.d $DOCKER_ENTRY_DIR/
 COPY	src/*/exit.d $DOCKER_EXIT_DIR/
+COPY	src/*/config $DOCKER_CONF_DIR1/
+COPY	src/*/plugin $DOCKER_PLUG_DIR/
 
 #
 # Install helpers. Set bash as default shell. Setup syslogs service.
@@ -46,7 +52,12 @@ RUN	apt-get update && apt-get install --yes --no-install-recommends \
 	gnupg \
 	jq \
 	inotify-tools \
-	&& docker-service.sh "syslogd -nO- -l$SYSLOG_LEVEL $SYSLOG_OPTIONS"
+	cron \
+	&& ln -s $DOCKER_CRONTAB_FILE $DOCKER_CRONTAB_DIR \
+	&& docker-service.sh \
+	"syslogd -nO- -l$SYSLOG_LEVEL $SYSLOG_OPTIONS" \
+	"cron -f"
+#	"cron -f -L 4"
 
 
 
@@ -88,7 +99,6 @@ ENV	DEBIAN_FRONTEND=noninteractive \
 	DOCKER_BIN_DIR=/usr/local/bin \
 	LMTP_LISTEN=*:2003 \
 	SA_GROUP=kopano \
-	LOG_METHOD=syslog \
 	DOCKER_BUILD_DEB_DIR=/tmp/deb \
 	DOCKER_BUILD_PASSES=1
 #
@@ -99,10 +109,15 @@ RUN	mkdir -p $DOCKER_BUILD_DEB_DIR \
 	https://download.kopano.io/community ${DIST} ${REL} ${ARCH}) \
 	&& echo "$webaddr<->${DIST} ${REL} ${ARCH}<-" \
 	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
+	&& webaddr=$(kopano-webaddr.sh archiver \
+	https://download.kopano.io/community ${DIST} ${REL} ${ARCH}) \
+	&& echo "$webaddr<->${DIST} ${REL} ${ARCH}<-" \
+	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
 	&& apt-get update \
 	&& for i in $(seq ${DOCKER_BUILD_PASSES}); do echo "\033[1;36mKOPANO CORE INSTALL PASS: $i\033[0m" \
 	&& dpkg --install --force-depends --skip-same-version --recursive $DOCKER_BUILD_DEB_DIR \
 	&& apt-get install --yes --no-install-recommends --fix-broken; done \
+	&& apt-get install --yes --no-install-recommends python3-ldap \
 	&& mkdir -p /var/lib/kopano/attachments && chown $DOCKER_APPL_RUNAS: /var/lib/kopano/attachments \
 	&& mkdir -p $DOCKER_APPL_SSL_DIR \
 	&& mkdir -p $DOCKER_ACME_SSL_DIR \
@@ -165,6 +180,14 @@ RUN	apt-get install --yes --no-install-recommends apache2 libapache2-mod-php \
 	https://download.kopano.io/community ${DIST} ${REL} all) \
 	&& echo "$webaddr<->${DIST} ${REL} all<-" \
 	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
+	&& webaddr=$(kopano-webaddr.sh mdm \
+	https://download.kopano.io/community ${DIST} ${REL} all) \
+	&& echo "$webaddr<->${DIST} ${REL} all<-" \
+	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
+	&& webaddr=$(kopano-webaddr.sh smime \
+	https://download.kopano.io/community ${DIST} ${REL} ${ARCH}) \
+	&& echo "$webaddr<->${DIST} ${REL} all<-" \
+	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
 	&& apt-get update \
 	&& for i in $(seq ${DOCKER_BUILD_PASSES}); do echo "\033[1;36mKOPANO WEBAPP INSTALL PASS: $i\033[0m" \
 	&& dpkg --install --force-depends --skip-same-version --recursive $DOCKER_BUILD_DEB_DIR \
@@ -173,7 +196,6 @@ RUN	apt-get install --yes --no-install-recommends apache2 libapache2-mod-php \
 	&& . docker-common.sh \
 	&& . docker-config.sh \
 	&& dc_replace /etc/kopano/webapp/config.php 'define("SECURE_COOKIES", true);' 'define("SECURE_COOKIES", false);' \
-#	&& dc_fixmissing /etc/php/7.?/apache2/conf.d/kopano.ini /etc/php/7.?/mods-available/kopano.ini /etc/php5/conf.d/kopano.ini \
 	&& dc_replace /etc/apache2/sites-available/kopano-webapp.conf 'Alias /webapp /usr/share/kopano-webapp' '<VirtualHost *:80>\\nDocumentRoot /usr/share/kopano-webapp' \
 	&& echo '</VirtualHost>' >> /etc/apache2/sites-available/kopano-webapp.conf \
 	&& dc_modify /etc/apache2/apache2.conf '^ErrorLog' syslog:user \
@@ -186,6 +208,7 @@ RUN	apt-get install --yes --no-install-recommends apache2 libapache2-mod-php \
 	&& a2dissite 000-default.conf \
 	&& a2ensite kopano-webapp \
 	&& rm -rf $DOCKER_BUILD_DEB_DIR \
+	&& cp -r $DOCKER_CONF_DIR1/webapp $DOCKER_SMPL_DIR1 \
 	&& docker-service.sh "-f -s /etc/apache2/envvars -q apache2 -DFOREGROUND -DNO_DETACH -k start"
 #
 # Ports
@@ -213,7 +236,7 @@ ENV	DEBIAN_FRONTEND=noninteractive \
 #
 # Add Z-Push repository and install Z-Push configured to be used with Kopano and Apache
 #
-RUN	debaddr="$(kopano-webaddr.sh --deb final http://repo.z-hub.io/z-push: ${DIST} ${REL})" \
+RUN	debaddr="$(kopano-webaddr.sh --deb final https://download.kopano.io/zhub/z-push: ${DIST} ${REL})" \
 	&& echo "deb $debaddr/ /" > /etc/apt/sources.list.d/z-push.list \
 	&& wget -qO - $debaddr/Release.key | apt-key add - \
 	&& mkdir -p /var/lib/z-push && chown www-data: /var/lib/z-push \

Makefile

@@ -1,17 +1,26 @@
+# Makefile
+#
+# build
+#
+
 -include    *.mk
 
-#BLD_ARG  ?= --build-arg DIST=ubuntu --build-arg REL=18.04 --build-arg ARCH=i386
-BLD_ARG  ?=
+#BLD_ARG  ?= --build-arg DIST=ubuntu --build-arg REL=20.04
 BLD_REPO ?= mlan/kopano
 BLD_VER  ?= latest
 BLD_TGT  ?= full
+
 SRC_CMD  ?= src/kopano/bin/kopano-webaddr.sh -VV
 SRC_VER  ?= $(shell $(SRC_CMD))
-IMG_REPO ?= $(BLD_REPO)
-IMG_VER  ?= $(BLD_VER)
+
+TST_REPO ?= $(BLD_REPO)
+TST_VER  ?= $(BLD_VER)
 TST_ENV  ?= -C test
 TST_TGTE ?= $(addprefix test-,all diff down env htop imap lmtp logs mail pop3 pull sh sv up)
 TST_TGTI ?= test_% test-up_%
+
+export TST_REPO TST_VER
+
 _version  = $(if $(findstring $(BLD_TGT),$(1)),\
 $(if $(findstring latest,$(2)),latest $(1) $(SRC_VER) $(1)-$(SRC_VER),$(2) $(1)-$(2)),\
 $(if $(findstring latest,$(2)),$(1) $(1)-$(SRC_VER),$(1)-$(2)))
@@ -34,7 +43,7 @@ ps:
 	docker ps -a
 
 prune:
-	docker image prune
+	docker image prune -f
 
 clean:
 	docker images | grep $(BLD_REPO) | awk '{print $$1 ":" $$2}' | uniq | xargs docker rmi

README.md

@@ -3,8 +3,9 @@
 ![travis-ci test](https://img.shields.io/travis/mlan/docker-kopano.svg?label=build&style=popout-square&logo=travis)
 ![docker build](https://img.shields.io/docker/cloud/build/mlan/kopano.svg?label=build&style=popout-square&logo=docker)
 ![image Size](https://img.shields.io/docker/image-size/mlan/kopano.svg?label=size&style=popout-square&logo=docker)
-![docker stars](https://img.shields.io/docker/stars/mlan/kopano.svg?label=stars&style=popout-square&logo=docker)
 ![docker pulls](https://img.shields.io/docker/pulls/mlan/kopano.svg?label=pulls&style=popout-square&logo=docker)
+![docker stars](https://img.shields.io/docker/stars/mlan/kopano.svg?label=stars&style=popout-square&logo=docker)
+![github stars](https://img.shields.io/github/stars/mlan/docker-kopano.svg?label=stars&style=popout-square&logo=github)
 
 This (non official) repository provides dockerized web mail service as well as Exchange ActiveSync (EAS), IMAP, POP3 and ICAL service (and their secure variants IMAPS, POP3S and ICALS). It is based on [Kopano](https://kopano.com) core components, as well as the Kopano WebApp and [Z-Push](http://z-push.org/). The image uses [nightly built packages](https://download.kopano.io/community/) which are provided by the Kopano community.
 
@@ -21,6 +22,8 @@ Hopefully this repository can be retired once the Kopano community make official
 - Configuration using environment variables
 - Log directed to docker daemon with configurable level
 - Built in utility script [run](src/docker/bin/run) helping configuring Kopano components, WebApp and Z-Push
+- [Move to public with LDAP lookup](#move-to-public-with-ldap-lookup)
+- [Crontab](https://en.wikipedia.org/wiki/Cron) support.
 - Health check
 - Hook for theming
 - Demo based on `docker-compose.yml` and `Makefile` files
@@ -64,14 +67,14 @@ services:
     image: mlan/kopano
     networks:
       - backend
-    ports:
-      - "127.0.0.1:8008:80"    # WebApp & EAS (alt. HTTP)
-      - "127.0.0.1:143:143"    # IMAP (not needed if all devices can use EAS)
-      - "127.0.0.1:110:110"    # POP3 (not needed if all devices can use EAS)
-      - "127.0.0.1:8080:8080"  # ICAL (not needed if all devices can use EAS)
-      - "127.0.0.1:993:993"    # IMAPS (not needed if all devices can use EAS)
-      - "127.0.0.1:995:995"    # POP3S (not needed if all devices can use EAS)
-      - "127.0.0.1:8443:8443"  # ICALS (not needed if all devices can use EAS)
+    ports:           # Expose ports to host interfaces
+      - "80:80"      # WebApp & EAS (alt. HTTP)
+      - "143:143"    # IMAP (not needed if all devices can use EAS)
+      - "110:110"    # POP3 (not needed if all devices can use EAS)
+      - "8080:8080"  # ICAL (not needed if all devices can use EAS)
+      - "993:993"    # IMAPS (not needed if all devices can use EAS)
+      - "995:995"    # POP3S (not needed if all devices can use EAS)
+      - "8443:8443"  # ICALS (not needed if all devices can use EAS)
     depends_on:
       - auth
       - db
@@ -81,9 +84,12 @@ services:
       - LDAP_URI=ldap://auth:389/
       - MYSQL_HOST=db
       - SMTP_SERVER=mta
-      - LDAP_SEARCH_BASE=${LDAP_BASE-dc=example,dc=com}
-      - LDAP_USER_TYPE_ATTRIBUTE_VALUE=${LDAP_USEROBJ-posixAccount}
-      - LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=${LDAP_GROUPOBJ-posixGroup}
+      - LDAP_SEARCH_BASE=${AD_BASE-dc=example,dc=com}
+      - LDAP_USER_TYPE_ATTRIBUTE_VALUE=${AD_USR_OB-kopano-user}
+      - LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=${AD_GRP_OB-kopano-group}
+      - LDAP_GROUPMEMBERS_ATTRIBUTE_TYPE=dn
+      - LDAP_PROPMAP=
+      - DAGENT_PLUGINS=movetopublicldap
       - MYSQL_DATABASE=${MYSQL_DATABASE-kopano}
       - MYSQL_USER=${MYSQL_USER-kopano}
       - MYSQL_PASSWORD=${MYSQL_PASSWORD-secret}
@@ -93,8 +99,9 @@ services:
       - IMAPS_LISTEN=*:993                      # enable TLS
       - POP3S_LISTEN=*:995                      # enable TLS
       - ICALS_LISTEN=*:8443                     # enable TLS
-      - DISABLED_FEATURES=${DISABLED_FEATURES-} # also enable IMAP and POP3
+      - PLUGIN_SMIME_USER_DEFAULT_ENABLE_SMIME=true
       - SYSLOG_LEVEL=${SYSLOG_LEVEL-3}
+      - LOG_LEVEL=${LOG_LEVEL-3}
     volumes:
       - app-conf:/etc/kopano
       - app-atch:/var/lib/kopano/attachments
@@ -109,15 +116,16 @@ services:
     hostname: ${MAIL_SRV-mx}.${MAIL_DOMAIN-example.com}
     networks:
       - backend
-    ports:
-      - "127.0.0.1:25:25"      # SMTP
+    ports:           # Expose ports to host interfaces
+      - "25:25"      # SMTP
+      - "465:465"    # SMTPS authentication required
     depends_on:
       - auth
     environment: # Virgin config, ignored on restarts unless FORCE_CONFIG given.
       - LDAP_HOST=auth
       - VIRTUAL_TRANSPORT=lmtp:app:2003
-      - LDAP_USER_BASE=ou=${LDAP_USEROU-users},${LDAP_BASE-dc=example,dc=com}
-      - LDAP_QUERY_FILTER_USER=(&(objectclass=${LDAP_USEROBJ-posixAccount})(mail=%s))
+      - LDAP_USER_BASE=ou=${AD_USR_OU-users},${AD_BASE-dc=example,dc=com}
+      - LDAP_QUERY_FILTER_USER=(&(objectclass=${AD_USR_OB-kopano-user})(mail=%s))
     volumes:
       - mta:/srv
       - app-spam:/var/lib/kopano/spamd          # kopano-spamd integration
@@ -144,8 +152,10 @@ services:
     image: mlan/openldap
     networks:
       - backend
+    command: --root-cn ${AD_ROOT_CN-admin} --root-pw ${AD_ROOT_PW-secret}
     environment:
-      - LDAP_LOGLEVEL=parse
+      - LDAPBASE=${AD_BASE-dc=example,dc=com}
+      - LDAPDEBUG=${AD_DEBUG-parse}
     volumes:
       - auth:/srv
       - /etc/localtime:/etc/localtime:ro        # Use host timezone
@@ -165,7 +175,7 @@ volumes:
 
 ## Demo
 
-This repository contains a [demo](demo) directory which hold the [docker-compose.yml](demo/docker-compose.yml) file as well as a [Makefile](demo/Makefile) which might come handy. Start with cloning the [github](https://github.com/mlan/docker-kopano) repository.
+This repository contains a [demo](demo) directory which hold the [docker-compose.yml](demo/docker-compose.yml) file as well as a [Makefile](demo/Makefile) which might come handy. To run the demo you need [docker-compose](https://docs.docker.com/compose/install/) installed. By default `curl` and `firefox` is expected to be installed, but if not, run `make utils-container` within the [demo](demo) directory once the repository has been cloned. The `make` utility works nicely with [bash-completion](https://github.com/scop/bash-completion) so it can be worth considering having it installed too. Once the dependencies are met, start with cloning the [github](https://github.com/mlan/docker-kopano) repository.
 
 ```bash
 git clone https://github.com/mlan/docker-kopano.git
@@ -177,7 +187,7 @@ From within the [demo](demo) directory you can start the containers by typing:
 make init
 ```
 
-Then you can assess WebApp on the URL [`http://localhost:8008`](http://localhost:8008) and log in with the user name `demo` and password `demo` . You can send yourself a test email by typing:
+Now you can assess WebApp on the custom docker network at URL `http://app` and log in with the user name `demo` and password `demo`.
 
 ```bash
 make web
@@ -189,7 +199,7 @@ You can send yourself a test email by typing:
 make test
 ```
 
-When you are done testing you can destroy the test containers by typing
+When you are done testing you can destroy the test containers and their volumes by typing:
 
 ```bash
 make destroy
@@ -212,17 +222,22 @@ The unlock file approach was selected since it is difficult to accidentally _cre
 
 In the rare event that want to modify the configuration of an existing container you can override the default behavior by setting `FORCE_CONFIG=overwrite` to a no-empty string.
 
-## Environment variables
+## Environment variables and service parameters
 
-When you create the `mlan/kopano` container, you can adjust the configuration of the Kopano server by passing one or more environment variables or on the docker run command line. Note that any pre-existing configuration files within the container will be left untouched.
+When you create the `mlan/kopano` container, you can adjust the configuration of the Kopano server by defining one or more environment variables. During container initiation environment variables are matched against all possible parameters for all Kopano services. When matched, configuration files are updated with the value of the matching environment variable.
 
-To see all available configuration variables you can run `man` within the container by for example using the [Makefile](demo/Makefile) described above:
+To see all available configuration parameters you can run `run list_parms <service>` within the container or when when using the [demo](#demo) described above just type:
 
 ```bash
-make mail-app-man_server
+make app-parms_dagent
+make app-parms_server
 ```
 
-If you do, you will notice that configuration variable names are all lower case, but they will be matched with all uppercase environment variables by the container `docker-entrypoint.sh` script.
+### Overlapping parameter names
+
+Some services use the same parameter names. When such a parameter is set using en environment variable all configuration files of the related services will be updated. This is not always desired. To address this you can prefix the parameter name with the name of the service you which to target by using the following syntax: `<service>_<parameter>`.
+
+For example when using the [Kopano-archiver](https://documentation.kopano.io/kopano_archiver_manual/) service, you often want to use one SQL database for the server and one separate one for the archiver. In this situation you can have two separate SQL database containers, one at `db-srv` and the other at `db-arc`. To set the [`MYSQL_HOST`](#mysql_host) parameter in the two relevant configuration files use `SERVER_MYSQL_HOST=db-srv` and `ARCHIVER_MYSQL_HOST=db-arc`. If for some reason you want both services to call the same container, instead use: `MYSQL_HOST=db`.
 
 ## SQL database configuration
 
@@ -260,27 +275,25 @@ When the `ATTACHMENT_STORAGE` option is `ATTACHMENT_STORAGE=files`, this option
 
 When creating the SQL container you can use environment variables to initiate it. For example, `MYSQL_ROOT_PASSWORD=topsecret`, `MYSQL_DATABASE=kopano`, `MYSQL_USER=kopano` and `MYSQL_PASSWORD=verysecret`.
 
-## User authentication `USER_PLUGIN`
+## User management `USER_PLUGIN`
 
-Kopano supports three different ways to manage user authentication. Use the `USER_PLUGIN` environment variable to select the source of the user base. Possible values are: `db` (default), `ldap` and `unix`.
+Kopano supports three different plugins for user management. Use the `USER_PLUGIN` environment variable to select the source of the user base. Possible values are: `db` (default), `ldap` and `unix`.
 
- `db`: Retrieve the users from the Kopano database. Use the kopano-admin tool to create users and groups. There are no additional settings for this plug-in.
+`db`: Retrieve the users from the Kopano database. Use the `kopano-admin` tool to create users and groups. There are no additional settings for this plug-in.
 
-`ldap`: Retrieve the users and groups information from an LDAP server. All additional LDAP settings are needed see below
+`ldap`: Retrieve the users and groups information from an LDAP directory server. Additional LDAP settings are needed, see below.
 
 `unix`: Retrieve the users and groups information from the Linux password files. This option is probably not interesting here.
 
-### LDAP authentication
+### Accessing an LDAP directory server
 
-An LDAP server with user accounts configured to be used with Kopano is needed, but how to set one up is out of our scope here, instead see: [Kopano Knowledge Base/Install and optimize OpenLDAP for use with Kopano Groupware Core](https://kb.kopano.io/display/WIKI/Install+and+optimize+OpenLDAP+for+use+with+Kopano+Groupware+Core).
+The `USER_PLUGIN=ldap` retrieves user information from an LDAP directory server. A brief description of how that is achieved is described in [Setup an LDAP directory server](#setup-an-ldap-directory-server). Once the LDAP directory server is up and running, the `mlan/kopano` container can be configured to use it using environment variables.
 
-Once the LDAP server is up and running, the `mlan/kopano` container can be configured to use it using environment variables. In addition to the variables discussed below also set `USER_PLUGIN=ldap`.
-
-#### `LDAP_URI`
+#### Host address `LDAP_URI`
 
 Specifies the URI of one or more LDAP server(s) to use, without any DN portion, such as `ldap://server:389/`, `ldaps://server:636/` or `ldapi:///`. Defaults: `LDAP_URI=ldap://localhost:389/`.
 
-The historic directives `LDAP_HOST`, `LDAP_PORT`, `LDAP_PROTOCOL` are no longer supported (8.7.85).
+Note that. the historic directives `LDAP_HOST`, `LDAP_PORT`, `LDAP_PROTOCOL` are no longer supported (8.7.85).
 
 #### `LDAP_SEARCH_BASE`
 
@@ -298,11 +311,19 @@ This variable determines what defines a valid Kopano group. Default: `LDAP_GROUP
 
 Adds an extra filter to the user search. Default `LDAP_USER_SEARCH_FILTER=`
 
-Hint: Use the `kopanoAccount` attribute in the filter to differentiate between non-kopano and kopano users.
+Hint: Use the `kopanoAccount` attribute in the filter to differentiate between non-Kopano and Kopano users.
+
+#### `LDAP_BIND_USER`, `LDAP_BIND_PASSWD`
+
+The defaults for these environment variables are empty. If you cannot bind anonymously, do it with this distinguished name and password. Example: LDAP_BIND_USER=cn=admin,dc=example,dc=com, LDAP_BIND_PASSWD=secret.
+
+### Kopano LDAP attributes `LDAP_PROPMAP`
+
+The Kopano services needs to know which of the users LDAP attributes, like addresses, phone numbers and company information, to use. This information is defined in the `propmap` file, which is included in the Kopano installation files here `/usr/share/kopano/ldap.propmap.cfg`. When using `USER_PLUGIN=ldap` this LDAP `propmap` file is used by the Kopano services by setting `LDAP_PROPMAP=` to an empty string. Optionally you can use another file, for example`LDAP_PROPMAP=/etc/kopano/ldap.propmap.cfg`. If no file can be found there the installed one will be copied there.
 
 ## Enabling IMAP, POP3 and ICAL
 
-By default the [IMAP](https://www.atmail.com/blog/imap-commands/) and POP3 services are disabled for all users. Set the environment variable `DISABLED_FEATURES=` to an empty string to enable both IMAP and POP3 for all users.
+By default the [IMAP](https://www.atmail.com/blog/imap-commands/) and POP3 services are disabled for all users. Set the environment variable `DISABLED_FEATURES=` to an empty string to enable both IMAP and POP3 for all users. You can override this setting for each user independently by enabling or disabling features in the LDAP directory server see, [Setup an LDAP directory server](#setup-an-ldap-directory-server).
 
 #### `DISABLED_FEATURES`
 
@@ -388,7 +409,15 @@ Separately, `LOG_LEVEL` controls the logging level of the Kopano services. `LOG_
 | ---- | ---- | ---- | ------- | ------ | ---- | ----- |
 | 0    | 1    | 2    | **3**   | 4      | 5    | 6     |
 
-## Custom themes
+## Kopano add-ons
+
+### Kopano Archiver
+
+The [Kopano Archiver](https://documentation.kopano.io/kopano_archiver_manual/) provides a Hierarchical Storage Management (HSM) solution for Kopano. With the Kopano Archiver older messages will be automatically moved to slower and thus cheaper storage. The slow storage consists of one or more additional Kopano Archive servers which sole task it is to store archived messages.
+
+Typically the archiver needs its own SQL database. You can configure it using environment variables. When you do, pay attention to [overlapping parameter names](#overlapping-parameter-names). Also the archiver does not run as a daemon but instead you can set up [cron](#cron) jobs. For example, to run the archiver daily with weakly clean-up you can use; `CRONTAB_ENTRY1=0 1 * * * root kopano-archiver -A` and `CRONTAB_ENTRY2=0 3 * * 0 root kopano-archiver -C`.
+
+## WebApp custom themes
 
 You can easily customize the Kopano WebApp see [New! JSON themes in Kopano WebApp](https://kopano.com/blog/new-json-themes-in-kopano-webapp/). Once you have the files you can install them in your docker container using the receipt below, where we assume that the container name is `mail-app` and that the directory `mytheme` contains the `theme.json` and the other file defining the theme.
 
@@ -400,6 +429,81 @@ docker exec -it mail-app run dc_replace /etc/kopano/webapp/config.php 'define("T
 
 Please note that it is not possible to rename the directory `/etc/kopano/theme/Custom` within the container without further modifications.
 
+## WebApp plugins
+
+### S/MIME
+
+[S/MIME](https://en.wikipedia.org/wiki/S/MIME) provides [email encryption](https://en.wikipedia.org/wiki/Email_encryption) guaranteeing the confidentiality and non-repudiation of email. The [S/MIME](https://documentation.kopano.io/webapp_smime_manual/) WebApp plugin is pre-installed.
+
+Using the [demo](#demo) you can easily create a S/MIME certificate you can try out using WebApp.
+
+```sh
+make app-create_smime
+```
+
+### Mobile device management
+
+The [Mobile Device Management](https://documentation.kopano.io/webapp_mdm_manual/) WebApp plugin comes pre-installed. With it you can resync, remove, refresh and even wipe your devices, connected via [Exchange ActiveSync (EAS)](https://en.wikipedia.org/wiki/Exchange_ActiveSync).
+
+## Public folders
+
+There are two type of stores (folders containing communication elements); private and public stores. There can only be one public store. It is the Kopano dagent that places incoming messages into mail boxes, that is the private and public stores.
+
+Public folders are configured by the system admin. Users have them mapped automatically. The shared and public folders can be synced with mobile devices via [Exchange ActiveSync (EAS)](https://wiki.z-hub.io/display/ZP/Sharing+folders+and+Read-only). Each user can manage which shared or public folders to sync with her mobile devices by using the [Mobile Device Management](#mobile-device-management) WebApp plugin.
+
+With the current Kopano implementation, delivering to the public store is configured separately from normal user management. There is the [move to public](https://documentation.kopano.io/kopanocore_administrator_manual/special_kc_configurations.html#move-to-public) plugin which moves incoming messages to a folder in the public store. It has a static configuration and does not support LDAP lookup. 
+
+### Move to public with LDAP lookup
+
+The `mlan/kopano` image include a extended version of the move to public plugin which use LDAP lookup, instead of a static file based lookup. When the plugin [move to public with LDAP](src/kopano/plugin/movetopublicldap.py) is enabled, `DAGENT_PLUGINS=movetopublicldap`, the kopano dagent will do two LDAP queries. The first is to search for an entry/user with matching email address. The second, introduced with this plugin, is to get the public folder from this entry. If found the message will be delivered to the public folder otherwise it will be delivered to the mailbox of the user.
+
+Lets demonstrate how delivery to a public folder is configured. With this LDAP entry:
+
+```yaml
+dn: uid=public,ou=users,dc=example,dc=com
+cn: public
+objectClass: top
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: public
+uid: public
+mail: public@example.com
+kopanoAccount: 1
+kopanoHidden: 1
+kopanoSharedStoreOnly: 1
+kopanoResourceType: publicFolder:Public Stores/public
+```
+
+messages to `public@example.com` will be delivered to the public store in `Public Stores/public`.
+The central [attribute](https://documentation.kopano.io/kopanocore_administrator_manual/appendix_b.html#appendix-b-ldap-attribute-description) is `kopanoResourceType: publicFolder:Public Stores/public`. It contains a token and a folder name. The token match is case sensitive and there must be a colon `:` separating
+the token and the public folder name. The folder name can contain space and
+sub folders, which are distinguished using a forward slash `/`.
+
+The parameters in `/etc/kopano/ldap.cfg` will be used to arrange the LDAP queries.
+The LDAP attribute holding the token and the token itself have the following
+default values, which can be modified in `/etc/kopano/movetopublicldap.cfg` if desired.
+
+```yaml
+ldap_public_folder_attribute = kopanoResourceType
+ldap_public_folder_attribute_token = publicFolder
+```
+
+As with other parameters, environment variables can be used to define them: `LDAP_PUBLIC_FOLDER_ATTRIBUTE=kopanoResourceType` and `LDAP_PUBLIC_FOLDER_ATTRIBUTE_TOKEN=publicFolder`.
+
+## Shared folders
+
+Users can share folders when sufficient permission have been granted. When logged into WebApp with an administrative account (`kopanoAdmin: 1`) you can modify the permissions on users shares and folders. Users can then, when logged into WebApp, open the inbox of other users by selecting `Open Shared Mails`.
+
+The [impersonation](https://wiki.z-hub.io/display/ZP/Impersonation) mechanism allow such shared folders to be synced over [Exchange ActiveSync (EAS)](https://wiki.z-hub.io/display/ZP/Sharing+folders+and+Read-only) too. Each user can manage which shared or public folders to sync with her mobile devices by using the [Mobile Device Management](#mobile-device-management) WebApp plugin.
+
+## Crontab
+
+The `mlan/kopano` has a [cron](https://en.wikipedia.org/wiki/Cron) service activated. You can use environment variables to set up [crontab](https://man7.org/linux/man-pages/man5/crontab.5.html) entries. Any environment variable name staring with `CRONTAB_ENTRY` will be use to add entries to cron.
+
+One trivial example is `CRONTAB_ENTRY_TEST=* * * * * root logger -t cron -p user.notice "SHELL=$$SHELL, PATH=$$PATH"`.
+
+During the initial configuration procedure any `CRONTAB_ENTRY` will add crontab entries to the file `/etc/kopano/docker-crontab`, all the while previously present entries are deleted. This file defines the `PATH` variable so that you don't need to give full path names to commands in the crontab entry. This is, you need to provide the full path names to commands if this `PATH` definition is missing in the `/etc/kopano/docker-crontab` file.
+
 ## Mail transfer agent interaction
 
 Environment variables can be used to configure where Kopano find the Mail Transfer Agent, such as Postfix. Likewise the Mail Transfer Agent need to know where to forward emails to.
@@ -441,10 +545,59 @@ Sometimes a new version of Kopano breaks compatibility with old configurations.
 
 Prior to Kopano WebApp version 5.0.0 the parameter was `define("INSECURE_COOKIES", true);` was used to allow HTTP access. Now [`define("SECURE_COOKIES", false);`](https://documentation.kopano.io/webapp_admin_manual/config.html#secure-cookies) is used instead. This fix tries to update the configuration accordingly.
 
+### `MIGRATE_CONFIG=2` Make sure WebApp plugins have configuration files in place
+
+The WebApp plugins S/MIME and MDM has recently been added to the `mlan/kopano` image. Old deployments might not have the related configuration files in place, preventing these plugins from running. This fix places default copies of configuration files in the configuration directory should they be missing.
+
 # Knowledge base
 
 Here some topics relevant for arranging a mail server are presented.
 
+## Setup an LDAP directory server
+
+A [Directory Server Agent (DSA)](https://en.wikipedia.org/wiki/Directory_System_Agent) is used to store, organize and present data in a key-value type format. Typically, directories are optimized for lookups, searches, and read operations over write operations, so they function extremely well for data that is referenced often but changes infrequently.
+
+The [Lightweight Directory Access Protocol (LDAP)](https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol), is an open protocol used to store and retrieve data from a hierarchical directory structure. Commonly used to store information about an organization and its assets and users.
+
+[OpenLDAP](https://www.openldap.org/) is a cross platform LDAP based directory service. [Active Directory (AD)](https://en.wikipedia.org/wiki/Active_Directory) is a [directory service](https://en.wikipedia.org/wiki/Directory_service) developed by [Microsoft](https://en.wikipedia.org/wiki/Microsoft) for [Windows domain](https://en.wikipedia.org/wiki/Windows_domain) networks which also uses LDAP.
+
+There are many dockerized OpenLDAP server to choose from. One such example is [mlan/openldap](https://github.com/mlan/docker-openldap).
+
+### LDAP user entry
+
+The data itself in an LDAP system is mainly stored in elements called attributes. Attributes are basically key-value pairs. Unlike in some other systems, the keys have predefined names which are dictated by the objectClasses selected for an entry. An entry is basically a collection of attributes under a name used to describe something; a user for example.
+
+```yaml
+dn: uid=demo,ou=users,dc=example,dc=com
+objectclass: inetOrgPerson
+uid: demo
+```
+
+### Kopano LDAP schema
+
+The Kopano LDAP schema defines additional objectClasses and attributes. These allow the Kopano services, access for example, to controlled on per-user basis. More information on available attributes can be find here, [Fine-tuning user configuration](https://documentation.kopano.io/kopanocore_administrator_manual/configure_kc_components.html?highlight=propmap#fine-tuning-user-configuration).
+
+```shell
+dn: uid=demo,ou=users,dc=example,dc=com
+objectClass: top
+objectClass: inetOrgPerson
+objectClass: kopano-user
+objectClass: posixAccount
+cn: demo
+sn: demo
+uid: demo
+mail: demo@example.com
+uidNumber: 1234
+gidNumber: 1234
+homeDirectory: /home/demo
+telephoneNumber: 0123 123456789
+title: MCP
+kopanoEnabledFeatures: imap
+kopanoEnabledFeatures: pop3
+```
+
+The schema needs to be added to the directory server. The Kopano installation files include the LDAP schema and can be found here `/usr/share/doc/kopano/kopano.ldif.gz`. For more details, see: [Kopano Knowledge Base/Install and optimize OpenLDAP for use with Kopano Groupware Core](https://kb.kopano.io/display/WIKI/Install+and+optimize+OpenLDAP+for+use+with+Kopano+Groupware+Core).
+
 ## Kopano WebApp HTTP access
 
 The distribution installation of `kopano-webapp` only allow HTTPS access. The `mlan/kopano` image updates the configuration to [`define("SECURE_COOKIES", false);`](https://documentation.kopano.io/webapp_admin_manual/config.html#secure-cookies) in `/etc/kopano/webapp/config.php` also allowing HTTP access. This can be useful when arranging the `mlan/kopano` container behind a reverse proxy, like [Traefik](https://doc.traefik.io/traefik/), which then does the enforcement of HTTPS. Also see [`MIGRATE_CONFIG=1` Rejected insecure request as configuration for SECURE_COOKIES is true](#migrate_config1-rejected-insecure-request-as-configuration-for-secure_cookies-is-true).

ROADMAP.md

@@ -1,54 +1,33 @@
 # Road map
 
-## kDAV
-
-Consider integrating support for kDAV which provides CalDAV and CardDAV.
-
-## Common configurations
-
-The following directives exist:
-
-```sh
-!include common.cfg
-```
-
 ## Revisit Persistent Data
 
 Consider consolidating directories which are candidates for persistence under `/srv`.
 
+- /etc/kopano
+- /var/lib/kopano
+- /var/lib/z-push
+
 ### Kopano Search
 
 The kopano-search module keeps its database here, /var/lib/kopano/search.
 Consider to also consolidating it under /srv to simplify making it persistent?
 
+## kDAV
+
+Consider integrating support for kDAV which provides CalDAV and CardDAV.
+
 ## webapp-passwd
 
 Integrate [webapp-passwd](https://github.com/silentsakky/zarafa-webapp-passwd)?
 
-## S/MIME
+## kopano-spamd and kopano-search logs
 
-Install and configure [S/MIME](https://kopano.com/blog/s-mime-plugin-description/)?
+In [KC-1858](https://github.com/Kopano-dev/kopano-core/commit/4a7f833e170167ebfa4f4c55835f8760ce7617f3) we find:
 
-[S/MIME manual](https://documentation.kopano.io/webapp_smime_manual/).
+> The syslog log method does not work correctly and thus this change
+> disables it. Until it is fixed, Python services do not support
+> the syslog log_method. Additionally an environment variable is
+> added, which allow to lift this restriction for testing when it
+> it set.
 
-## MDM
-
-Install and configure [MDM](https://documentation.kopano.io/webapp_mdm_manual/)?
-With the MDM plugin you can resync, remove, refresh and even wipe your device.
-
-## Improve Health Check?
-
-Verify the user anonymously.
-```bash
-ldapsearch -h dockerhost -xLLL -b dc=example,dc=com '(kopanoAccount=1)'
-```
-
-Check if kopano can get the user from LDAP
-```bash
-kopano-admin -l
-```
-check that apache and mysql is running
-```bash
-apache2ctl status
-mysqlcheck -A
-```

demo/.env

@@ -1,22 +1,34 @@
 COMPOSE_PROJECT_NAME=demo
-SYSLOG_LEVEL=6
+SYSLOG_LEVEL=7
 LOG_LEVEL=6
+AD_DEBUG=stats
+AD_ADM_CN=admin
+AD_ADM_PW=admin
+AD_ADM_TEL=555-540-9637
+AD_ADM_TIT=System Admin
+AD_BASE=dc=example,dc=com
+AD_GRP_CN=team
+AD_GRP_OB=kopano-group
+AD_GRP_OU=groups
+AD_PUB_CN=public
+AD_ROOT_CN=admin
+AD_ROOT_PW=secret
+AD_SHR_CN=shared
+AD_USR_AS=trial
+AD_USR_CN=demo
+AD_USR_OB=kopano-user
+AD_USR_OU=users
+AD_USR_PW=demo
+AD_USR_TEL=555-439-2736
+AD_USR_TIT=First User
+DKIM_SELECTOR=default
 MAIL_DOMAIN=example.com
 MAIL_SRV=mx
-REGEX_ALIAS='/([^+]+)[+-].*@(.+)/ $1@$2'
-DKIM_SELECTOR=default
-SA_TAG_LEVEL_DEFLT=-999
-SA_DEBUG=0
-RAZOR_REGISTRATION=
-LDAP_BASE=dc=example,dc=com
-LDAP_USEROU=users
-LDAP_USEROBJ=posixAccount
-LDAP_USERFLT=
-LDAP_GROUPOU=groups
-LDAP_GROUPOBJ=posixGroup
-LDAP_TEST_USER=demo
-LDAP_TEST_PASSWD=demo
-MYSQL_ROOT_PASSWORD=secret
 MYSQL_DATABASE=kopano
-MYSQL_USER=kopano
 MYSQL_PASSWORD=secret
+MYSQL_ROOT_PASSWORD=secret
+MYSQL_USER=kopano
+RAZOR_REGISTRATION=
+REGEX_ALIAS='/([^+]+)[+-].*@(.+)/ $1@$2'
+SA_DEBUG=0
+SA_TAG_LEVEL_DEFLT=-999

demo/.gitignore

@@ -0,0 +1,2 @@
+ssl
+utils-container.mk

demo/Makefile

@@ -1,19 +1,46 @@
+# Makefile
+#
+# demo
+#
+
 -include    *.mk .env .init.env
 
-srv_list ?= auth app db mta
-curl_dbg ?= -v -s
-_ip       = $(shell docker inspect -f \
-	'{{range .NetworkSettings.Networks}}{{println .IPAddress}}{{end}}' \
-	$(1) | head -n1)
+SRV_LIST ?= auth app db mta
 
-.PHONY:
+AD_DOM   ?= $(call ad_sub_dot, $(AD_BASE))
+AD_DC    ?= $(call ad_cut_dot, 1, 1, $(AD_DOM))
+
+SSL_O     = $(MAIL_DOMAIN)
+SSL_MAIL  = auto
+SSL_PASS  = $(AD_USR_PW)
+#SSL_TRST  = $(SSL_SMIME)
+
+NET_NAME ?= $(COMPOSE_PROJECT_NAME)_backend
+CURL_OPT ?= -s -v
+TSSL_CMD ?= docker run -i --rm --network $(NET_NAME) drwetter/testssl.sh
+CURL_CMD ?= curl
+webb_cmd ?= firefox $(1) &
+APP_NAME  = app
+AUT_NAME  = auth
+AUW_NAME  = auth-web
+DB_NAME   = db
+DBW_NAME  = db-web
+MTA_NAME  = mta
+APP_FQDN ?= $(call dkr_srv_ip,$(APP_NAME))
+AUT_FQDN ?= $(call dkr_srv_ip,$(AUT_NAME))
+AUW_FQDN ?= $(call dkr_cnt_ip,$(AUW_NAME))
+DB_FQDN  ?= $(call dkr_srv_ip,$(DB_NAME))
+DBW_FQDN ?= $(call dkr_cnt_ip,$(DBW_NAME))
+MTA_FQDN ?= $(call dkr_srv_ip,$(MTA_NAME))
+
+MAIL_FROM ?= test@my-domain.biz
 
 variables:
 	make -pn | grep -A1 "^# makefile"| grep -v "^#\|^--" | sort | uniq
 
 test: all-test_quiet mta-test_smtp
 
-init: auth-init db-init mta-init app-init
+init: up auth-init db-init app-restart mta-init app-init
 
 ps:
 	docker-compose ps
@@ -24,7 +51,7 @@ up:
 down:
 	docker-compose down
 
-destroy:
+destroy: auth-web-down db-web-down all-destroy_smime
 	docker-compose down -v
 
 config:
@@ -36,52 +63,192 @@ logs:
 images:
 	docker-compose images
 
-$(addsuffix -up,$(srv_list)):
+$(addsuffix -up,$(SRV_LIST)):
 	docker-compose up -d $(patsubst %-up,%,$@)
 
-$(addsuffix -down,$(srv_list)):
-	docker-compose rm -s $(patsubst %-down,%,$@)
+$(addsuffix -down,$(SRV_LIST)):
+	docker-compose rm -sf $(patsubst %-down,%,$@)
 
-$(addsuffix -restart,$(srv_list)):
+$(addsuffix -restart,$(SRV_LIST)):
 	docker-compose restart $(patsubst %-restart,%,$@)
 
-$(addsuffix -renew,$(srv_list)):
+$(addsuffix -renew,$(SRV_LIST)):
 	docker-compose rm -s $(patsubst %-renew,%,$@)
 	docker-compose up -d $(patsubst %-renew,%,$@)
 
-$(addsuffix -top,$(srv_list)):
+$(addsuffix -top,$(SRV_LIST)):
 	docker-compose top $(patsubst %-top,%,$@)
 
-$(addsuffix -logs,$(srv_list)):
+$(addsuffix -logs,$(SRV_LIST)):
 	docker-compose logs $(patsubst %-logs,%,$@)
 
-$(addsuffix -sh,$(srv_list)):
+$(addsuffix -pull,$(SRV_LIST)):
+	docker-compose pull $(patsubst %-pull,%,$@)
+
+$(addsuffix -sh,$(SRV_LIST)):
 	docker-compose exec $(patsubst %-sh,%,$@) sh -c 'exec $$(getent passwd root | sed "s/.*://g")'
 
-$(addsuffix -env,$(srv_list)):
+$(addsuffix -env,$(SRV_LIST)):
 	docker-compose exec $(patsubst %-env,%,$@) env
 
-$(addsuffix -sv,$(srv_list)):
+$(addsuffix -sv,$(SRV_LIST)):
 	docker-compose exec $(patsubst %-sv,%,$@) sh -c 'sv status $$SVDIR/*'
 
-$(addsuffix -diff,$(srv_list)):
+$(addsuffix -diff,$(SRV_LIST)):
 	docker container diff $(COMPOSE_PROJECT_NAME)_$(patsubst %-diff,%,$@)_1
 
 wait_%:
 	sleep $*
 
-web:
-	firefox localhost:8008 &
+web: app-web
 
-auth-init: auth-up wait_11 auth-mod_hash auth-mod_index auth-add_user
+auth-init: wait_3 auth-mod_conf auth-add_schema auth-add_data
+
+export define LDIF_MOD_CONF
+dn: olcDatabase={-1}frontend,cn=config
+changetype: modify
+add: olcPasswordHash
+olcPasswordHash: {CRYPT}
+
+dn: cn=config
+changetype: modify
+add: olcPasswordCryptSaltFormat
+olcPasswordCryptSaltFormat: $$6$$%.16s
+
+dn: olcDatabase={1}mdb,cn=config
+changetype: modify
+add: olcDbIndex
+olcDbIndex: cn,ou,uid,mail eq
+endef
+
+export define LDIF_ADD_DATA
+dn: $(AD_BASE)
+objectClass: organization
+objectClass: dcObject
+dc: $(AD_DC)
+o: $(AD_DOM)
+
+dn: ou=$(AD_USR_OU),$(AD_BASE)
+ou: $(AD_USR_OU)
+objectClass: organizationalUnit
+
+dn: ou=$(AD_GRP_OU),$(AD_BASE)
+ou: $(AD_GRP_OU)
+objectClass: organizationalUnit
+
+dn: cn=$(AD_GRP_CN),ou=$(AD_GRP_OU),$(AD_BASE)
+cn: $(AD_GRP_CN)
+objectClass: groupOfNames
+objectClass: kopano-group
+member: uid=$(AD_ADM_CN),ou=$(AD_USR_OU),$(AD_BASE)
+member: uid=$(AD_USR_CN),ou=$(AD_USR_OU),$(AD_BASE)
+mail: $(AD_GRP_CN)@$(MAIL_DOMAIN)
+
+dn: uid=$(AD_ADM_CN),ou=$(AD_USR_OU),$(AD_BASE)
+changetype: add
+cn: $(AD_ADM_CN)
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: $(AD_ADM_CN)
+uid: $(AD_ADM_CN)
+mail: $(AD_ADM_CN)@$(MAIL_DOMAIN)
+userPassword: $(AD_ADM_PW)
+telephoneNumber: $(AD_ADM_TEL)
+title: $(AD_ADM_TIT)
+kopanoAccount: 1
+kopanoAdmin: 1
+kopanoEnabledFeatures: imap
+kopanoEnabledFeatures: pop3
+
+dn: uid=$(AD_USR_CN),ou=$(AD_USR_OU),$(AD_BASE)
+changetype: add
+cn: $(AD_USR_CN)
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: $(AD_USR_CN)
+uid: $(AD_USR_CN)
+mail: $(AD_USR_CN)@$(MAIL_DOMAIN)
+userPassword: $(AD_USR_PW)
+telephoneNumber: $(AD_USR_TEL)
+title: $(AD_USR_TIT)
+kopanoAccount: 1
+kopanoAliases: $(AD_USR_AS)@$(MAIL_DOMAIN)
+kopanoEnabledFeatures: imap
+kopanoEnabledFeatures: pop3
+
+dn: uid=$(AD_SHR_CN),ou=$(AD_USR_OU),$(AD_BASE)
+cn: $(AD_SHR_CN)
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: $(AD_SHR_CN)
+uid: $(AD_SHR_CN)
+mail: $(AD_SHR_CN)@$(MAIL_DOMAIN)
+kopanoAccount: 1
+kopanoSharedStoreOnly: 1
+
+dn: uid=$(AD_PUB_CN),ou=$(AD_USR_OU),$(AD_BASE)
+cn: $(AD_PUB_CN)
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: $(AD_PUB_CN)
+uid: $(AD_PUB_CN)
+mail: $(AD_PUB_CN)@$(MAIL_DOMAIN)
+kopanoAccount: 1
+kopanoHidden: 1
+kopanoSharedStoreOnly: 1
+kopanoResourceType: publicFolder:Public Stores/public
+endef
+
+define smtp_mail
+@printf "From: <$(2)>\nTo: <$(3)>\nDate: $$(date -R)\nSubject: $(4)\
+\n\nGreat news! You can receive email.\n" | tee /dev/tty \
+| $(CURL_CMD) $(1) -T - --mail-from $(2) --mail-rcpt $(3) $(CURL_OPT)
+endef
+
+define lmtp_mail
+printf "LHLO mx\nMAIL FROM: <$(2)>\nRCPT TO: <$(3)>\nDATA\
+\nFrom: <$(2)>\nTo: <$(3)>\nDate: $$(date -R)\nSubject: $(4)\
+\n\nGreat news! You can receive email.\n.\nQUIT\n" | tee /dev/tty \
+| $(CURL_CMD) $(1) -T - $(CURL_OPT)
+endef
+
+export define MAKE_UTILS_CONTAINER
+CURL_CMD ?= docker run -i --rm --network $(NET_NAME) curlimages/curl
+webb_cmd ?= docker run -d --rm --network $(NET_NAME) \
+-e DISPLAY=$$$$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix \
+-v /etc/localtime:/etc/localtime:ro -v $$$$(pwd)/ssl:/ssl \
+kennethkl/firefox $$(1)
+APP_FQDN ?= $(APP_NAME)
+AUT_FQDN ?= $(AUT_NAME)
+AUW_FQDN ?= $(AUW_NAME)
+DB_FQDN  ?= $(DB_NAME)
+DBW_FQDN ?= $(DBW_NAME)
+MTA_FQDN ?= $(MTA_NAME)
+endef
+
+utils-container:
+	echo "$$MAKE_UTILS_CONTAINER" > utils-container.mk
+
+utils-default:
+	rm -f utils-container.mk
+
+auth-mod_conf:
+	echo "$$LDIF_MOD_CONF" | docker-compose exec -T auth ldapmodify -Q
+
+auth-add_data:
+	echo "$$LDIF_ADD_DATA" | docker-compose exec -T auth ldapadd -Q
+
+auth-add_schema:
+	docker-compose exec app zcat /usr/share/doc/kopano/kopano.ldif.gz \
+	| docker-compose exec -T auth ldapadd -Q
 
 auth-show_conf:
-	docker-compose exec auth ldap search -b cn=config "(cn=config)"
-	docker-compose exec auth ldap search -b cn=config olcDatabase={-1}frontend
-	docker-compose exec auth ldap search -b cn=config olcDatabase={1}mdb
+	docker-compose exec auth ldapsearch -QLLLb cn=config "(cn=config)"
+	docker-compose exec auth ldapsearch -QLLLb cn=config olcDatabase={-1}frontend
+	docker-compose exec auth ldapsearch -QLLLb cn=config olcDatabase={1}mdb
 
-auth-show_user:
-	docker-compose exec auth ldap search -b "$(LDAP_BASE)"
+auth-show_data:
+	docker-compose exec auth ldapsearch -QLLL
 
 auth-show_cat0:
 	docker-compose exec auth slapcat -n0
@@ -89,44 +256,44 @@ auth-show_cat0:
 auth-show_cat1:
 	docker-compose exec auth slapcat -n1
 
-auth-add_user:
-	printf "dn: ou=$(LDAP_USEROU),$(LDAP_BASE)\nchangetype: add\nobjectClass: organizationalUnit\nobjectClass: top\nou: $(LDAP_USEROU)\n\ndn: ou=$(LDAP_GROUPOU),$(LDAP_BASE)\nchangetype: add\nobjectClass: organizationalUnit\nobjectClass: top\nou: $(LDAP_GROUPOU)\n\ndn: uid=$(LDAP_TEST_USER),ou=$(LDAP_USEROU),$(LDAP_BASE)\nchangetype: add\nobjectClass: top\nobjectClass: inetOrgPerson\nobjectClass: $(LDAP_USEROBJ)\ncn: $(LDAP_TEST_USER)\nsn: $(LDAP_TEST_USER)\nuid: $(LDAP_TEST_USER)\nmail: $(LDAP_TEST_USER)@$(MAIL_DOMAIN)\nuidNumber: 1234\ngidNumber: 1234\nhomeDirectory: /home/$(LDAP_TEST_USER)\nuserPassword: $(LDAP_TEST_PASSWD)\n" \
-	| docker-compose exec -T auth ldap modify
+auth-web: auth-web-up
+	sleep 2
+	$(call webb_cmd,http://$(AUW_FQDN))
 
-auth-mod_index:
-	printf "dn: olcDatabase={1}mdb,cn=config\nchangetype: modify\nadd: olcDbIndex\nolcDbIndex: cn,ou,uid,mail eq\n" \
-	| docker-compose exec -T auth ldap modify
+auth-web-up:
+	docker run -d --name $(AUW_NAME) --network $(NET_NAME) \
+	-e PHPLDAPADMIN_LDAP_HOSTS=auth -e PHPLDAPADMIN_HTTPS=false \
+	osixia/phpldapadmin || true
 
-auth-mod_hash:
-	printf "dn: olcDatabase={-1}frontend,cn=config\nchangetype: modify\nadd: olcPasswordHash\nolcPasswordHash: {CRYPT}\n\ndn: cn=config\nchangetype: modify\nadd: olcPasswordCryptSaltFormat\nolcPasswordCryptSaltFormat: \$$6\$$%%.16s\n" \
-	| docker-compose exec -T auth ldap modify
+auth-web-down:
+	docker rm -f $(AUW_NAME) || true
 
-mta-init: mta-up
+mta-init:
 
 mta-bayes:
 	docker-compose exec mta sh -c 'rm -f bayesian.database.gz && wget http://artinvoice.hu/spams/bayesian.database.gz && gunzip bayesian.database.gz && sa-learn --restore bayesian.database && chown -R amavis: /var/amavis/.spamassassin && rm -rf bayesian.database'
 
 mta-test_smtp:
-	printf "From: A tester <test@example.biz>\nTo: <$(LDAP_TEST_USER)@$(MAIL_DOMAIN)>\nDate: $$(date)\nSubject: A SMTP test message\n\nGreat news! You can receive email.\n" \
-	| curl smtp://localhost -T - --mail-from test@example.biz \
-	--mail-rcpt $(LDAP_TEST_USER)@$(MAIL_DOMAIN) $(curl_dbg)
+	$(call smtp_mail,smtp://$(MTA_FQDN),$(MAIL_FROM),$(AD_USR_CN)@$(MAIL_DOMAIN),A SMTP test message.)
+
+mta-test_regexp:
+	$(call smtp_mail,smtp://$(MTA_FQDN),$(MAIL_FROM),$(AD_USR_CN)+info@$(MAIL_DOMAIN),A regexp SMTP test message.)
 
 mta-test_smtps:
-	printf "From: A tester <test@example.biz>\nTo: <$(LDAP_TEST_USER)@$(MAIL_DOMAIN)>\nDate: $$(date)\nSubject: A SMTPS test message\n\nGreat news! You can receive secure email.\n" \
-	| curl smtps://localhost -T - --mail-from test@example.biz -k \
-	-u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD) \
-	--mail-rcpt $(LDAP_TEST_USER)@$(MAIL_DOMAIN) $(curl_dbg)
+	$(call smtp_mail,smtps://$(MTA_FQDN),$(MAIL_FROM),$(AD_USR_CN)@$(MAIL_DOMAIN),A secure SMTPS test message.) \
+	-k -u $(AD_USR_CN):$(AD_USR_PW)
 
-mta-test_smtp2:
-	printf "From: A info tester <test-info@example.biz>\nTo: <$(LDAP_TEST_USER)-info@$(MAIL_DOMAIN)>\nDate: $$(date)\nSubject: A SMTP test message \n\nGreat news! $(LDAP_TEST_USER)-info@$(MAIL_DOMAIN) can also receive email.\n" \
-	| curl smtp://localhost -T - --mail-from test@example.biz \
-	--mail-rcpt $(LDAP_TEST_USER)@$(MAIL_DOMAIN) $(curl_dbg)
+mta-test_shared: all-test_quiet
+	$(call smtp_mail,smtp://$(MTA_FQDN),$(MAIL_FROM),$(AD_SHR_CN)@$(MAIL_DOMAIN),A shared SMTP test message.)
+
+mta-test_public: all-test_quiet
+	$(call smtp_mail,smtp://$(MTA_FQDN),$(MAIL_FROM),$(AD_PUB_CN)@$(MAIL_DOMAIN),A public SMTP test message.)
 
 mta-razor:
 	docker-compose exec mta run amavis_register_razor
 
 mta-apk_list:
-	docker-compose exec mta /bin/sh -c 'for pkg in $$(apk info 2>/dev/null); do printf "%9s  %s\n" $$(apk info -s $$pkg 2>/dev/null | sed -n "2{p;q}") $$pkg; done | sort'
+	docker-compose exec mta /bin/sh -c 'for pkg in $$(apk info 2>/dev/null); do printf "%9s %3s %s\n" $$(apk info -s $$pkg 2>/dev/null | sed -n "2{p;q}") $$pkg; done | sort | sort -k 2,2'
 
 mta-quarantine_list:
 	docker-compose exec mta amavis-ls
@@ -137,11 +304,11 @@ mta-freshclam_nodns:
 mta-clamdtop:
 	docker-compose exec mta clamdtop
 
-mta-debugtools:
+mta-tools:
 	docker-compose exec mta apk --no-cache --update add \
-	nano less lsof htop openldap-clients bind-tools iputils strace
+	nano less lsof htop openldap-clients bind-tools iputils strace iproute2
 
-mta-htop: mta-debugtools
+mta-htop: mta-tools
 	docker-compose exec mta htop
 
 mta-encrypt:
@@ -160,92 +327,112 @@ mta-show_mailq:
 mta-flush_mailq:
 	docker-compose exec mta postqueue -f
 
-mta-hostaddr:
-	$(eval myhost := $(call _ip,$(COMPOSE_PROJECT_NAME)_mta_1))
-
 mta-test_auth:
-	docker-compose exec mta doveadm auth test $(LDAP_TEST_USER) $(LDAP_TEST_PASSWD)
+	docker-compose exec mta doveadm auth test $(AD_USR_CN) $(AD_USR_PW)
 
-mta-test_imap: mta-hostaddr
-	curl imap://$(myhost) -X CAPABILITY
-	curl imap://$(myhost) -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD)
+mta-test_imaps:
+	$(CURL_CMD) imaps://$(MTA_FQDN)//inbox -X "fetch 1 all" \
+	--ssl --anyauth -k -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
 
-mta-test_rimap:
-	docker-compose exec mta curl imap://app -X CAPABILITY
-	docker-compose exec mta curl imap://app -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD)
+mta-man:
+	docker-compose exec mta apk --no-cache --update add man-db man-pages \
+	postfix-doc cyrus-sasl-doc dovecot-doc spamassassin-doc clamav-doc razor-doc
 
-mta-test_ldap: mta-debugtools
-	docker-compose exec mta ldapsearch -H ldap://auth:389 -xLLL -s base namingContexts
-
-db-init: db-up
+db-init:
 
 db-test:
 	docker-compose exec db mysqlshow -u $(MYSQL_USER) $(MYSQL_DATABASE) -p$(MYSQL_PASSWORD)
 
-app-init: app-up wait_12 app-public_store
+db-web: db-web-up
+	sleep 2
+	$(call webb_cmd,http://$(DBW_FQDN))
 
-app-debugtools:
+db-web-up:
+	docker run -d --name $(DBW_NAME) --network $(NET_NAME) \
+	-e PMA_HOST=db phpmyadmin/phpmyadmin || true
+
+db-web-down:
+	docker rm -f $(DBW_NAME) || true
+
+app-init: app-public_store app-create_smime
+
+app-tools:
 	docker-compose exec app apt-get update
 	docker-compose exec app apt-get install --yes \
-	less nano ldap-utils htop net-tools lsof iputils-ping strace
+	less nano ldap-utils htop net-tools lsof iputils-ping dnsutils strace
 
-app-htop: app-debugtools
+app-htop: app-tools
 	docker-compose exec app htop
 
-app-man_server:
-	docker-compose exec app man kopano-server.cfg
-
-app-man_ldap:
-	docker-compose exec app man kopano-ldap.cfg
-
-app-hostaddr:
-	$(eval myhost := $(call _ip,$(COMPOSE_PROJECT_NAME)_app_1))
-
 app-test_smtp: mta-test_smtp
 
-app-test_lmtp: app-hostaddr
-	printf "LHLO mx\nMAIL FROM: <test@example.biz>\nRCPT TO: <$(LDAP_TEST_USER)@$(MAIL_DOMAIN)>\nDATA\nFrom: A tester <test@example.biz>\nTo: <$(LDAP_TEST_USER)@$(MAIL_DOMAIN)>\nDate: $$(date)\nSubject: A LMTP test message from me to you\n\nDelete me, please \n.\nQUIT\n" | nc -C $(myhost) 2003
+app-test_lmtp:
+	$(call lmtp_mail,telnet://$(APP_FQDN):2003,$(MAIL_FROM),$(AD_USR_CN)@$(MAIL_DOMAIN),A LMTP test message.)
 
-app-test_all: all-test_muted app-test_imap app-test_pop3 app-test_ical app-test_imaps app-test_pop3s app-test_icals
+app-test_all: all-test_muted $(addprefix app-test_,imap pop3 ical imaps pop3s icals)
 
-app-test_imap: app-hostaddr
-	curl imap://$(myhost) -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD) $(curl_dbg)
+app-test_imap:
+	$(CURL_CMD) imap://$(APP_FQDN) -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
 
-app-test_imaps: app-hostaddr
-	curl imaps://$(myhost) -k -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD) $(curl_dbg)
+app-test_imaps:
+	$(CURL_CMD) imaps://$(APP_FQDN) -k -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
 
-app-test_pop3: app-hostaddr
-	curl pop3://$(myhost) -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD) $(curl_dbg)
+app-test_pop3:
+	$(CURL_CMD) pop3://$(APP_FQDN) -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
 
-app-test_pop3s: app-hostaddr
-	curl pop3s://$(myhost) -k -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD) $(curl_dbg)
+app-test_pop3s:
+	$(CURL_CMD) pop3s://$(APP_FQDN) -k -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
 
-app-test_ical: app-hostaddr
-	curl http://$(myhost):8080 -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD) $(curl_dbg)
+app-test_ical:
+	$(CURL_CMD) http://$(APP_FQDN):8080 -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
 
-app-test_icals: app-hostaddr
-	curl https://$(myhost):8443 -k -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD) $(curl_dbg)
+app-test_icals:
+	$(CURL_CMD) https://$(APP_FQDN):8443 -k -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
 
-app-test_tls: app-hostaddr
-	docker run --rm -it --network demo_backend drwetter/testssl.sh app:993 || true
+app-test_tls:
+	$(TSSL_CMD) $(APP_FQDN):993 || true
+
+app-web:
+	$(call webb_cmd,http://$(APP_FQDN))
+
+app-test_oof1:
+	docker-compose exec app kopano-oof -u $(AD_USR_CN) -m 1 -t "Dunno when I return"
+
+app-test_oof0:
+	docker-compose exec app kopano-oof -u $(AD_USR_CN) -m 0
 
 app-show_user1:
-	docker-compose exec app kopano-admin -l
+	docker-compose exec app kopano-admin --details $(AD_USR_CN)
 
-app-show_user2: app-debugtools
-	docker-compose exec app ldapsearch -H ldap://auth:389 -xLLL -b $(LDAP_BASE) '*'
+app-show_user2: app-tools
+	docker-compose exec app ldapsearch -H ldap://auth:389 -xLLL -b $(AD_BASE) '*'
 
 app-show_sync:
 	docker-compose exec app z-push-top
 
 app-create_store:
-	docker-compose exec app kopano-admin --create-store $(LDAP_TEST_USER)
+	docker-compose exec app kopano-admin --create-store $(AD_USR_CN)
 
 app-public_store:
-	docker-compose exec app kopano-storeadm -h default: -P
+	docker-compose exec app kopano-storeadm -P
+
+#app-add_user:
+#	docker-compose exec app kopano-admin -c $(AD_USR_CN) -p $(AD_USR_PW) \
+#	-e $(AD_USR_CN)@$(MAIL_DOMAIN) -f $(AD_USR_CN) -a 1
+
+$(addprefix app-parms_,archiver dagent gateway ical ldap search server spamd spooler):
+	docker-compose exec app run list_parms $(patsubst app-parms_%,%,$@)
+
+app-create_smime: all-create_smime
+	docker cp ssl/ca.crt $(call dkr_srv_cnt,app):/usr/local/share/ca-certificates/$(MAIL_DOMAIN)_CA.crt
+	docker-compose exec app update-ca-certificates
 
 all-test_quiet:
-	$(eval curl_dbg := -s -S )
+	$(eval CURL_OPT := -s -S )
 
 all-test_muted:
-	$(eval curl_dbg := -s -S >/dev/null || true)
+	$(eval CURL_OPT := -s -S >/dev/null || true)
+
+all-create_smime: ssl/$(AD_USR_CN).p12
+
+all-destroy_smime: ssl-destroy

demo/ad.mk

@@ -0,0 +1 @@
+../test/ad.mk

demo/dkr.mk

@@ -0,0 +1 @@
+../test/dkr.mk

demo/docker-compose.yml

@@ -5,26 +5,29 @@ services:
     image: mlan/kopano
     networks:
       - backend
-    ports:
-      - "127.0.0.1:8008:80"    # WebApp & EAS (alt. HTTP)
-      - "127.0.0.1:143:143"    # IMAP (not needed if all devices can use EAS)
-      - "127.0.0.1:110:110"    # POP3 (not needed if all devices can use EAS)
-      - "127.0.0.1:8080:8080"  # ICAL (not needed if all devices can use EAS)
-      - "127.0.0.1:993:993"    # IMAPS (not needed if all devices can use EAS)
-      - "127.0.0.1:995:995"    # POP3S (not needed if all devices can use EAS)
-      - "127.0.0.1:8443:8443"  # ICALS (not needed if all devices can use EAS)
+#    ports: # Uncomment to expose ports to host interfaces
+#      - "80:80"    # WebApp & EAS (alt. HTTP)
+#      - "143:143"    # IMAP (not needed if all devices can use EAS)
+#      - "110:110"    # POP3 (not needed if all devices can use EAS)
+#      - "8080:8080"  # ICAL (not needed if all devices can use EAS)
+#      - "993:993"    # IMAPS (not needed if all devices can use EAS)
+#      - "995:995"    # POP3S (not needed if all devices can use EAS)
+#      - "8443:8443"  # ICALS (not needed if all devices can use EAS)
     depends_on:
       - auth
       - db
       - mta
     environment: # Virgin config, ignored on restarts unless FORCE_CONFIG given.
       - USER_PLUGIN=ldap
-      - LDAP_URI=ldap://auth:389/
+      - LDAP_URI=ldap://auth/
       - MYSQL_HOST=db
       - SMTP_SERVER=mta
-      - LDAP_SEARCH_BASE=${LDAP_BASE-dc=example,dc=com}
-      - LDAP_USER_TYPE_ATTRIBUTE_VALUE=${LDAP_USEROBJ-posixAccount}
-      - LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=${LDAP_GROUPOBJ-posixGroup}
+      - LDAP_SEARCH_BASE=${AD_BASE-dc=example,dc=com}
+      - LDAP_USER_TYPE_ATTRIBUTE_VALUE=${AD_USR_OB-kopano-user}
+      - LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=${AD_GRP_OB-kopano-group}
+      - LDAP_GROUPMEMBERS_ATTRIBUTE_TYPE=dn
+      - LDAP_PROPMAP=
+      - DAGENT_PLUGINS=movetopublicldap
       - MYSQL_DATABASE=${MYSQL_DATABASE-kopano}
       - MYSQL_USER=${MYSQL_USER-kopano}
       - MYSQL_PASSWORD=${MYSQL_PASSWORD-secret}
@@ -34,8 +37,9 @@ services:
       - IMAPS_LISTEN=*:993                      # enable TLS
       - POP3S_LISTEN=*:995                      # enable TLS
       - ICALS_LISTEN=*:8443                     # enable TLS
-      - DISABLED_FEATURES=${DISABLED_FEATURES-} # also enable IMAP and POP3
+      - PLUGIN_SMIME_USER_DEFAULT_ENABLE_SMIME=true
       - SYSLOG_LEVEL=${SYSLOG_LEVEL-3}
+      - LOG_LEVEL=${LOG_LEVEL-3}
     volumes:
       - app-conf:/etc/kopano
       - app-atch:/var/lib/kopano/attachments
@@ -50,9 +54,9 @@ services:
     hostname: ${MAIL_SRV-mx}.${MAIL_DOMAIN-example.com}
     networks:
       - backend
-    ports:
-      - "127.0.0.1:25:25"      # SMTP
-      - "127.0.0.1:465:465"    # SMTPS authentication required
+#    ports: # # Uncomment to expose ports to host interfaces
+#      - "25:25"      # SMTP
+#      - "465:465"    # SMTPS authentication required
     depends_on:
       - auth
     environment: # Virgin config, ignored on restarts unless FORCE_CONFIG given.
@@ -63,8 +67,8 @@ services:
       - SMTP_TLS_SECURITY_LEVEL=${SMTP_TLS_SECURITY_LEVEL-}
       - SMTP_TLS_WRAPPERMODE=${SMTP_TLS_WRAPPERMODE-no}
       - SMTPD_USE_TLS=yes
-      - LDAP_USER_BASE=ou=${LDAP_USEROU-users},${LDAP_BASE-dc=example,dc=com}
-      - LDAP_QUERY_FILTER_USER=(&(objectclass=${LDAP_USEROBJ-posixAccount})(mail=%s))
+      - LDAP_USER_BASE=ou=${AD_USR_OU-users},${AD_BASE-dc=example,dc=com}
+      - LDAP_QUERY_FILTER_USER=(&(objectclass=${AD_USR_OB-kopano-user})(mail=%s))
       - LDAP_QUERY_ATTRS_PASS=uid=user
       - REGEX_ALIAS=${REGEX_ALIAS-}
       - DKIM_SELECTOR=${DKIM_SELECTOR-default}
@@ -99,8 +103,10 @@ services:
     image: mlan/openldap
     networks:
       - backend
+    command: --root-cn ${AD_ROOT_CN-admin} --root-pw ${AD_ROOT_PW-secret}
     environment:
-      - LDAP_LOGLEVEL=parse
+      - LDAPBASE=${AD_BASE-dc=example,dc=com}
+      - LDAPDEBUG=${AD_DEBUG-parse}
     volumes:
       - auth:/srv
       - /etc/localtime:/etc/localtime:ro        # Use host timezone

demo/ssl.mk

@@ -0,0 +1 @@
+../test/ssl.mk

src/acme/entry.d/10-acme-common

@@ -44,11 +44,13 @@ acme_monitor_tls_cert() {
 acme_symlink_tls_cert() {
 	if ([ -r $DOCKER_ACME_SSL_H_CERT ] && [ -r $DOCKER_ACME_SSL_H_KEY ]); then
 		dc_log 5 "Setting up ACME TLS certificate for host $HOSTNAME"
+		mkdir -p $DOCKER_APPL_SSL_DIR
 		ln -sf $DOCKER_ACME_SSL_H_CERT $DOCKER_APPL_SSL_CERT
 		ln -sf $DOCKER_ACME_SSL_H_KEY $DOCKER_APPL_SSL_KEY
 	else
 		if ([ -r $DOCKER_ACME_SSL_D_CERT ] && [ -r $DOCKER_ACME_SSL_D_KEY ]); then
 			dc_log 5 "Setting up ACME TLS certificate for domain $DOMAIN"
+			mkdir -p $DOCKER_APPL_SSL_DIR
 			ln -sf $DOCKER_ACME_SSL_D_CERT $DOCKER_APPL_SSL_CERT
 			ln -sf $DOCKER_ACME_SSL_D_KEY $DOCKER_APPL_SSL_KEY
 		fi

src/acme/entry.d/50-acme-monitor-tlscert

@@ -7,7 +7,7 @@
 #
 
 #
-# Setup ACME monitor and arrange certificate sym-links
+# Setup ACME monitor and arrange certificate symbolic links
 #
 acme_monitor_tls_cert
 acme_symlink_tls_cert

src/docker/bin/docker-config.sh

@@ -11,6 +11,8 @@ HOSTNAME=${HOSTNAME-$(hostname)}
 DOMAIN=${HOSTNAME#*.}
 TLS_KEYBITS=${TLS_KEYBITS-2048}
 TLS_CERTDAYS=${TLS_CERTDAYS-30}
+DOCKER_CRONTAB_FILE=${DOCKER_CRONTAB_FILE-/etc/crontab}
+DOCKER_CRONTAB_ENV=${DOCKER_CRONTAB_ENV-CRONTAB_ENTRY}
 
 #
 # general file manipulation commands, used both during build and run time
@@ -232,6 +234,18 @@ dc_prune_pidfiles() {
 	done
 }
 
+#
+# Setup crontab entries
+#
+dc_crontab_entries() {
+	local entries="$(eval echo \${!$DOCKER_CRONTAB_ENV*})"
+	for entry in $entries; do
+		[ -z "${changed+x}" ] && local changed= && sed -i '/^\s*[0-9*]/d' $DOCKER_CRONTAB_FILE
+		echo "${!entry}" >> $DOCKER_CRONTAB_FILE
+		dc_log 5 "Added entry ${!entry} in $DOCKER_CRONTAB_FILE"
+	done
+}
+
 #
 # TLS/SSL Certificates [openssl]
 #

src/docker/bin/docker-runfunc.sh

@@ -2,13 +2,14 @@
 #
 # docker-runfunc.sh
 #
-# Allow functions to be accessed from the commandline.
+# Allow functions to be accessed from the command line.
 #
 
 #
 # Source common functions.
 #
 . docker-common.sh
+. docker-config.sh
 
 #
 # dr_docker_call_func "$@"
@@ -17,7 +18,7 @@ dr_docker_call_func() {
 	export DOCKER_RUNFUNC="$@"
 	local cmd=$1
 	shift
-	dc_log 7 "CMD:$cmd ARG:$@"
+#	dc_log 7 "CMD:$cmd ARG:$@"
 	$cmd "$@"
 	exit 0
 }
@@ -28,7 +29,7 @@ dr_docker_call_func() {
 #
 dr_docker_run_parts() {
 	for file in $(find $1 -type f -name "$2" -executable 2>/dev/null|sort); do
-		dc_log 7 run_parts: executing $file
+#		dc_log 7 run_parts: executing $file
 		. $file
 	done
 }

src/docker/config/docker-crontab

@@ -0,0 +1,7 @@
+# /etc/cron.d/docker-crontab: crontab entries for docker
+
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+
+# During initialization CRONTAB_ENTRY envvars will overwrite all cron commands
+# in this file. Cron commands start with * or digits with optional white space.
+

src/docker/entry.d/20-docker-print-versions

@@ -1,5 +1,5 @@
 #!/bin/sh
 #
-# 10-docker-print-versions
+# 20-docker-print-versions
 #
 dc_pkg_versions kopano-common kopano-webapp z-push-kopano

src/docker/entry.d/50-docker-crontab-entry

@@ -0,0 +1,15 @@
+#!/bin/sh
+#
+# 50-docker-crontab-entry
+#
+# Functions defined in:
+# docker-config.sh
+#
+#
+
+#
+# Add crontab entries if the config is unlocked.
+#
+if dc_is_unlocked; then
+	dc_crontab_entries
+fi

src/kopano/bin/kopano-webaddr.sh

@@ -8,8 +8,8 @@ _webroot="https://download.kopano.io/community"
 _debroot="http://repo.z-hub.io/z-push:"
 _component="core"
 _stage="final"
-_dist="debian"
-_rel="9"
+_dist="ubuntu"
+_rel="20.04"
 _arch="amd64"
 
 #

src/kopano/entry.d/10-kopano-common

@@ -14,87 +14,157 @@
 #
 DOCKER_CONF_DIR1=${DOCKER_CONF_DIR1-/etc/kopano}
 DOCKER_CONF_DIR2=${DOCKER_CONF_DIR2-/usr/share/z-push}
+DOCKER_MAN5_DIR=${DOCKER_MAN5_DIR-/usr/share/man/man5/}
 DOCKER_APPL_SSL_CERT=${DOCKER_APPL_SSL_CERT-$DOCKER_APPL_SSL_DIR/cert.pem}
 DOCKER_APPL_SSL_KEY=${DOCKER_APPL_SSL_KEY-$DOCKER_APPL_SSL_DIR/priv_key.pem}
-
-dagent_cfg_file=$DOCKER_CONF_DIR1/dagent.cfg
-gateway_cfg_file=$DOCKER_CONF_DIR1/gateway.cfg
-ical_cfg_file=$DOCKER_CONF_DIR1/ical.cfg
-ldap_cfg_file=$DOCKER_CONF_DIR1/ldap.cfg
-search_cfg_file=$DOCKER_CONF_DIR1/search.cfg
-server_cfg_file=$DOCKER_CONF_DIR1/server.cfg
-spamd_cfg_file=$DOCKER_CONF_DIR1/spamd.cfg
-spooler_cfg_file=$DOCKER_CONF_DIR1/spooler.cfg
-
+DOCKER_LDAP_PMAP_FILE=${DOCKER_LDAP_PMAP_FILE-/usr/share/kopano/ldap.propmap.cfg}
+DOCKER_LDAP_SERVICES=${DOCKER_LDAP_SERVICES-archiver dagent gateway ical ldap search server spamd spooler}
 sqlstate_cfg_file=$DOCKER_CONF_DIR2/backend/sqlstatemachine/config.php
 zpush_cfg_file=$DOCKER_CONF_DIR2/config.php
+webapp_cfg_file=$DOCKER_CONF_DIR1/webapp/config.php
+webapp_mdm_cfg_file=$DOCKER_CONF_DIR1/webapp/config-mdm.php
+webapp_smime_cfg_file=$DOCKER_CONF_DIR1/webapp/config-smime.php
+
 
 #
-# Define environment variables
+# CLI commands
 #
-ssl_env_vars="SSL_CERTIFICATE_FILE SSL_CIPHERS SSL_CURVES SSL_PREFER_SERVER_CIPHERS SSL_PRIVATE_KEY_FILE SSL_VERIFY_CLIENT SSL_VERIFY_FILE SSL_VERIFY_PATH TLS_MIN_PROTO"
-dagent_env_vars="LMTP_LISTEN SPAM_HEADER_NAME SPAM_HEADER_VALUE LOG_LEVEL"
-gateway_env_vars="BYPASS_AUTH DISABLE_PLAINTEXT_AUTH HTML_SAFETY_FILTER IMAP_CAPABILITY_IDLE IMAP_EXPUNGE_ON_DELETE IMAP_IGNORE_COMMAND_IDLE IMAP_LISTEN IMAP_MAX_FAIL_COMMANDS IMAP_MAX_MESSAGESIZE IMAP_ONLY_MAILFOLDERS IMAP_PUBLIC_FOLDERS IMAPS_LISTEN POP3_LISTEN POP3S_LISTEN DISABLE_PLAINTEXT_AUTH LOG_LEVEL"
-ical_env_vars="ICAL_LISTEN ICALS_LISTEN ENABLE_ICAL_GET LOG_LEVEL"
-ldap_env_vars="LDAP_URI LDAP_STARTTLS LDAP_BIND_USER LDAP_BIND_PASSWD LDAP_SEARCH_BASE LDAP_USER_TYPE_ATTRIBUTE_VALUE LDAP_GROUP_TYPE_ATTRIBUTE_VALUE LDAP_USER_SEARCH_FILTER"
-search_env_vars="INDEX_PROCESSES INDEX_DRAFTS INDEX_JUNK SUGGESTIONS INDEX_ATTACHMENTS INDEX_ATTACHMENT_MAX_SIZE LOG_LEVEL"
-server_env_vars="MYSQL_HOST MYSQL_PORT MYSQL_DATABASE MYSQL_USER MYSQL_PASSWORD DISABLED_FEATURES USER_PLUGIN SEARCH_TIMEOUT LOG_LEVEL"
-spamd_env_vars="SPAM_DIR SPAM_DB HEADER_TAG LEARN_HAM HAM_DIR SA_GROUP LOG_LEVEL LOG_METHOD"
-spooler_env_vars="SMTP_SERVER SMTP_PORT LOG_LEVEL"
-
-sqlstate_env_vars="STATE_SQL_ENGINE STATE_SQL_SERVER STATE_SQL_PORT STATE_SQL_DATABASE STATE_SQL_USER STATE_SQL_PASSWORD STATE_SQL_OPTIONS"
-zpush_env_vars="TIMEZONE USE_CUSTOM_REMOTE_IP_HEADER USE_FULLEMAIL_FOR_LOGIN STATE_MACHINE STATE_DIR LOGBACKEND LOGLEVEL LOGAUTHFAIL LOG_SYSLOG_PROGRAM LOG_SYSLOG_FACILITY SYNC_CONFLICT_DEFAULT PING_INTERVAL FILEAS_ORDER SYNC_MAX_ITEMS UNSET_UNDEFINED_PROPERTIES ALLOW_WEBSERVICE_USERS_ACCESS USE_PARTIAL_FOLDERSYNC"
-
-#
-# Define helpers
-#
-
-_kopano_apply_envvars_gen() {
-	# move existing cfg files
-	local cfg_file=$1
-	shift
-	local env_vars=$@
-	if [ -e $cfg_file ]; then
-		mv -f $cfg_file $cfg_file.orig
-	fi
-	for env_var in $env_vars; do
-		if [ -n "${!env_var+x}" ]; then
-			dc_log 5 "Setting ${env_var,,} = ${!env_var} in $cfg_file"
-			echo ${env_var,,} = ${!env_var} >> $cfg_file
-		fi
+list_parms() {
+	local services="$DOCKER_LDAP_SERVICES $DAGENT_PLUGINS"
+	[ $# -ge 0 ] && services="$@"
+	for service in $services; do
+		[ $# -ne 1 ] && echo "# $service"
+		local man_file=$(kopano_gen_filename_man $service)
+		kopano_get_envvars_man $man_file
+		kopano_get_envvars_ext $service
+		[ $# -ne 1 ] && echo
 	done
 }
 
-_kopano_apply_phpenvvars_gen() {
-	local cfg_file=$1
-	shift
-	local env_vars=$@
+#
+# Apply environment variables to configuration files.
+# Use all valid keys (variables) for a service to see if there is a envvar with
+# identical name, if so apply its value to the config file.
+# With kopano-core use man page files to lookup valid keys.
+# With kopano-webapp and z-push use installed config file to find valid keys.
+#
+kopano_apply_envvars_core() {
+	kopano_enable_envvars_plugin
+	for service in $DOCKER_LDAP_SERVICES $DAGENT_PLUGINS; do
+		kopano_apply_envvars_cfg $service
+	done
+}
+
+kopano_apply_envvars_webapp() {
+	kopano_apply_envvars_php $webapp_cfg_file
+	kopano_apply_envvars_php $webapp_mdm_cfg_file
+	kopano_apply_envvars_php $webapp_smime_cfg_file
+}
+
+kopano_apply_envvars_zpush() {
+	kopano_apply_envvars_php $sqlstate_cfg_file
+	kopano_apply_envvars_php $zpush_cfg_file
+}
+
+#
+# kopano_apply_envvars_cfg <service>
+#
+kopano_apply_envvars_cfg() {
+	local service=$1
+	local cfg_file=$(kopano_gen_filename_cfg $service)
+	local man_file=$(kopano_gen_filename_man $service)
+	local env_vars="$(kopano_get_envvars_ext $service)"
+	if [ -f $man_file ]; then
+		env_vars="$(kopano_get_envvars_man $man_file) $env_vars"
+	else
+		dc_log 6 "Could not find $man_file"
+	fi
 	if [ -e $cfg_file ]; then
+		mv -f $cfg_file $cfg_file.bak
+	fi
+	for env_var in $env_vars; do
+		kopano_set_envvars_cfg $service $env_var
+	done
+}
+
+kopano_apply_envvars_php() {
+	local cfg_file=$1
+	if [ -e $cfg_file ]; then
+		local env_vars="$(kopano_get_envvars_php $cfg_file)"
 		for env_var in $env_vars; do
 			if [ -n "${!env_var}" ]; then
+				[ -z "${changed+x}" ] && local changed= && cp -f $cfg_file $cfg_file.bak
 				dc_log 5 "Setting ${env_var} = ${!env_var} in $cfg_file"
-				sed -ri "s/(\s*define).+${env_var}.+/\1\(\x27${env_var}\x27, \x27${!env_var}\x27\);/g" $cfg_file
+				sed -ri "s/(\s*define[('\"]+${env_var}['\",]+).+/\1 ${!env_var});/Ig" $cfg_file
 			fi
 		done
 	fi
 }
 
-kopano_apply_envvars() {
-	_kopano_apply_envvars_gen $dagent_cfg_file   $dagent_env_vars
-	_kopano_apply_envvars_gen $gateway_cfg_file  $gateway_env_vars $ssl_env_vars
-	_kopano_apply_envvars_gen $ical_cfg_file     $ical_env_vars    $ssl_env_vars
-	_kopano_apply_envvars_gen $ldap_cfg_file     $ldap_env_vars
-	_kopano_apply_envvars_gen $search_cfg_file   $search_env_vars
-	_kopano_apply_envvars_gen $server_cfg_file   $server_env_vars
-	_kopano_apply_envvars_gen $spamd_cfg_file    $spamd_env_vars
-	_kopano_apply_envvars_gen $spooler_cfg_file  $spooler_env_vars
+#
+# kopano_set_envvars_cfg <service> <envvar name> [explicit parameter name]
+#
+kopano_set_envvars_cfg() {
+	local service=$1
+	local cfg_file=$(kopano_gen_filename_cfg $service)
+	local env_var=$2
+	local uniq_var=${service^^}_${env_var}
+	local cfg_par="${3-$env_var =}"
+	if [ -n "${!uniq_var+x}" ]; then
+		local env_val="${!uniq_var}"
+	elif [ -n "${!env_var+x}" ]; then
+		local env_val="${!env_var}"
+	fi
+	if [ -n "${env_val+x}" ]; then
+		dc_log 5 "Setting ${cfg_par,,} ${env_val} in $cfg_file"
+		echo ${cfg_par,,} ${env_val} >> $cfg_file
+	fi
 }
 
-kopano_apply_phpenvvars() {
-	_kopano_apply_phpenvvars_gen $sqlstate_cfg_file $sqlstate_env_vars
-	_kopano_apply_phpenvvars_gen $zpush_cfg_file    $zpush_env_vars
+#
+# External parameters
+# services without a man file or additional parameters are listed here.
+#
+kopano_get_envvars_ext() {
+	local service=$1
+	local vars
+	case $service in
+		movetopublicldap)
+			vars="ldap_public_folder_attribute ldap_public_folder_attribute_token"
+			;;
+	esac
+	echo $vars | tr "[:lower:]" "[:upper:]" | tr " " "\n" | sort -u
 }
 
+#
+# Python plugins
+#
+kopano_enable_envvars_plugin() {
+	for service in dagent spooler; do
+		local envvar=${service^^}_PLUGINS
+		for plugin in ${!envvar}; do
+			local pyfile=$(find /usr/share/kopano* -name ${plugin}.py)
+			if [ -z "$pyfile" ]; then
+				dc_log 5 "Unable to locate ${service} python plugin ${plugin}"
+			else
+				dc_log 5 "Enabling ${service} python plugin ${plugin}"
+				mkdir -p $DOCKER_CONF_DIR1/${service}
+				ln -sf ${pyfile} $DOCKER_CONF_DIR1/${service}
+				export ${service^^}_PLUGIN_PATH=$DOCKER_CONF_DIR1/${service}
+				export ${service^^}_PLUGIN_ENABLED=yes
+			fi
+		done
+	done
+}
+
+#
+# Helpers
+#
+kopano_gen_filename_man() { echo $DOCKER_MAN5_DIR/kopano-$1.cfg.5.gz ;}
+kopano_gen_filename_cfg() { echo $DOCKER_CONF_DIR1/$1.cfg ;}
+kopano_get_envvars_man() { zcat $1 | sed -r "/^\.SS/!d;{s/^\.SS (.*)/\U\1/g;s/,//g}" | sort -u ;}
+kopano_get_envvars_php() { sed -nr "/define\(/Is/.*define\(['\"](.*)['\"], .*/\1/Ip" $1 | sort -u ;}
+
 #
 # Update SSL_CERTIFICATE_FILE and SSL_PRIVATE_KEY_FILE.
 #
@@ -121,6 +191,25 @@ kopano_generate_tls_cert() {
 	done
 	if ([ -z "$SSL_CERTIFICATE_FILE" ] && [ -n "$secure" ] && dc_is_installed openssl); then
 		dc_log 4 "$secure, but no certs given, so generating self-signed cert for host $HOSTNAME"
+		mkdir -p $DOCKER_APPL_SSL_DIR
 		dc_tls_setup_selfsigned_cert $DOCKER_APPL_SSL_CERT $DOCKER_APPL_SSL_KEY
 	fi
 }
+
+#
+# LDAP directives
+#
+kopano_apply_envvar_propmap() {
+	local service=ldap
+	local cfg_file=$(kopano_gen_filename_cfg $service)
+	local env_var=LDAP_PROPMAP
+	if [ -n "${!env_var+x}" ]; then
+		if [ -z "${!env_var}" ]; then
+			typeset "$env_var=$DOCKER_LDAP_PMAP_FILE"
+		elif [ ! -e "${!env_var}" ]; then
+			dc_log 5 "No ${!env_var} so copying $DOCKER_LDAP_PMAP_FILE there."
+			cp -f $DOCKER_LDAP_PMAP_FILE ${!env_var}
+		fi
+		kopano_set_envvars_cfg $service $env_var '!propmap'
+	fi
+}

src/kopano/entry.d/30-kopano-migrate

@@ -1,6 +1,6 @@
 #!/bin/sh
 #
-# 20-kopano-migrate
+# 30-kopano-migrate
 #
 # Try to make configs compatible with new version if MIGRATE_CONFIG is defined.
 # Set MIGRATE_CONFIG=1 2 3 to list of fixes or MIGRATE_CONFIG=all to attempt all fixes.
@@ -8,9 +8,16 @@
 kopano_apply_migrate_fixes() {
 	local applied
 	if [ -n "$MIGRATE_CONFIG" ]; then
-		for fix in ${MIGRATE_CONFIG/all/1}; do # list all fixes here
+		for fix in ${MIGRATE_CONFIG/all/1 2}; do # list all fixes here
 		case $fix in
 		1) dc_replace /etc/kopano/webapp/config.php 'define("INSECURE_COOKIES", true);' 'define("SECURE_COOKIES", false);' ;;
+		2)
+			for plugin in smime mdm; do
+				local cfg_file=$DOCKER_CONF_DIR1/webapp/config-$plugin.php
+				local src_file=$DOCKER_SMPL_DIR1/webapp/config-$plugin.php
+				[ ! -f "$cfg_file" ] && cp "$src_file" "$cfg_file"
+			done
+			;;
 		*) fix= ;;
 		esac
 		if [ -n "$fix" ]; then

src/kopano/entry.d/50-kopano-config

@@ -14,6 +14,8 @@ if dc_is_unlocked; then
 	kopano_export_tls_cert
 	kopano_generate_tls_cert
 	kopano_export_tls_cert
-	kopano_apply_envvars
-	kopano_apply_phpenvvars
+	kopano_apply_envvars_core
+	kopano_apply_envvars_webapp
+	kopano_apply_envvars_zpush
+	kopano_apply_envvar_propmap
 fi

src/kopano/plugin/movetopublicldap.cfg

@@ -0,0 +1,32 @@
+# movetopublicldap.cfg
+#
+# Move mail to public ldap configuration
+#
+# This configuration file is used by the dagent plugin 'movetopublicldap.py'
+# The default location is '/etc/kopano/movetopublicldap.cfg'
+#
+
+#
+# The plugin reads two coniguration files.
+# First it looks for parameters in '/etc/kopano/ldap.cfg' allowing
+# common conifuration parameters to be kept in on place.
+# Second it reads '/etc/kopano/movetopublicldap.cfg'
+#
+
+#
+# Parameters that can be set in both files:
+#
+# ldap_uri =
+# ldap_starttls = no
+# ldap_search_base =
+# ldap_bind_user =
+# ldap_bind_passwd =
+# ldap_user_unique_attribute = uid
+#
+
+#
+# Parameters that can only be set in ovetopublicldap.cfg:
+#
+# ldap_public_folder_attribute = kopanoResourceType
+# ldap_public_folder_attribute_token = publicFolder
+#

src/kopano/plugin/movetopublicldap.py

@@ -0,0 +1,229 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+""" (c) 2019 Kopano
+
+movetopublicldap.py
+
+This is an LDAP lookup extension to the move to public plugin movetopublic.py.
+The changes to the original work are highlihgted below.
+
+The move to public plugin moves incoming messages to a folder in the public
+store. If folders are missing they will be created.
+
+A LDAP entry including:
+
+kopanoAccount: 1
+kopanoResourceType: publicFolder:<public folder>
+
+will have its email delivered to the public store in <public folder>.
+The token match is case sensitive and there must be a colon ':' separating
+the token and the public folder name. The folder name can contain space and
+sub folders, which are distinguished using forward slash '/'.
+So if we have 'kopanoResourceType: publicFolder:Public Stores/public' emails will
+be delivered to 'Public Folders/Public Stores/public'.
+
+The parameters in /etc/kopano/ldap.cfg will be used for the LDAP query.
+The LDAP attribute holding the token and the token itself have the following
+default values, which can be modified in /etc/kopano/movetopublicldap.cfg
+if desired.
+
+ldap_public_folder_attribute = kopanoResourceType
+ldap_public_folder_attribute_token = publicFolder
+
+"""
+from sys import hexversion
+from MAPI.Util import GetPublicStore
+from MAPI.Struct import NEWMAIL_NOTIFICATION
+from MAPI import MAPI_UNICODE, MAPI_MODIFY, OPEN_IF_EXISTS, MDB_WRITE
+from MAPI.Tags import (PR_RECEIVED_BY_EMAIL_ADDRESS_W, PR_EC_COMPANY_NAME_W,
+	PR_IPM_PUBLIC_FOLDERS_ENTRYID, PR_ENTRYID, PR_MAILBOX_OWNER_ENTRYID,
+	IID_IMessage, IID_IExchangeManageStore)
+from plugintemplates import IMapiDAgentPlugin, MP_CONTINUE, MP_STOP_SUCCESS
+from zconfig import ZConfigParser
+import configparser
+import ldap
+import os.path
+
+class KConfigParser(ZConfigParser):
+	""" Extends zconfig.ZConfigParser to also allow !directive in cfg files.
+	Change type addition. """
+	def __init__(self, configfile, defaultconfig={}):
+		self.config = configparser.ConfigParser(defaults=defaultconfig,
+			delimiters=('='), comment_prefixes=('#', '!'), allow_no_value=True)
+		self.readZConfig(configfile)
+
+class MoveToPublic(IMapiDAgentPlugin):
+
+	prioPreDelivery = 50
+
+	config = {}
+
+	CONFIGFILES = ['/etc/kopano/ldap.cfg', '/etc/kopano/movetopublicldap.cfg']
+
+	DEFAULTCONFIG = {
+		'ldap_uri': None,
+		'ldap_starttls': "no",
+		'ldap_search_base': None,
+		'ldap_bind_user': None,
+		'ldap_bind_passwd': None,
+		'ldap_user_unique_attribute': "uid",
+		'ldap_public_folder_attribute': "kopanoResourceType",
+		'ldap_public_folder_attribute_token': "publicFolder"
+	}
+
+	def __init__(self, logger):
+		IMapiDAgentPlugin.__init__(self, logger)
+		self.readconfig(self.CONFIGFILES, self.DEFAULTCONFIG)
+
+	def readconfig(self, configfiles=CONFIGFILES, defaultconfig=DEFAULTCONFIG):
+		""" Reads ldap.cfg and movetopublicldap.cfg into self.config.
+		Change type addition. """
+		options = [opt.split('_', 1)[1] for opt in defaultconfig.keys()]
+		config = None
+		for configfile in configfiles:
+			if os.path.isfile(configfile):
+				self.logger.logDebug("*--- Reading file {}".format(configfile))
+				if not config:
+					config = KConfigParser(configfile, defaultconfig)
+				else:
+					config = KConfigParser(configfile, config.options())
+			else:
+				self.logger.logDebug("*--- Can't find file {}".format(configfile))
+		self.config = config.getdict('ldap',options)
+		self.logger.logDebug("*--- Config list {}".format(self.config))
+		return self.config
+
+	def searchfilter(self, recipient):
+		""" (&(uid=recipient)(kopanoResourceType=publicFolder:*)).
+		Change type addition. """
+		return ("(&({}={})({}={}:*))"
+			.format(self.config['user_unique_attribute'],
+			recipient,
+			self.config['public_folder_attribute'],
+			self.config['public_folder_attribute_token']))
+
+	def searchquery(self, recipient):
+		""" Query a LDAP/AD driectory server to lookup recipient using
+		search_base and return public_folder_attribute.
+		Change type addition. """
+		if not self.config or not self.config['uri']:
+			self.logger.logError(("!--- ldap_uri is not defined."
+				" Please check {}" .format(self.CONFIGFILES)))
+			return None
+		else:
+			l = ldap.initialize(self.config['uri'])
+			try:
+				l.protocol_version = ldap.VERSION3
+				if self.config['starttls'] == 'yes':
+					l.start_tls_s()
+				l.simple_bind_s(self.config['bind_user'] or u'', \
+					self.config['bind_passwd'] or u'')
+			except ldap.SERVER_DOWN as e:
+				self.logger.logError(("!--- LDAP server is not reachable {}"
+				.format(e)))
+				return None
+			except ldap.INVALID_CREDENTIALS as e:
+				self.logger.logError(("!--- Invalid LDAP credentials {}"
+					" Please check {}" .format(e, self.CONFIGFILES)))
+				l.unbind_s()
+				return None
+			except ldap.LDAPError as e:
+				self.logger.logError("!--- LDAPError {}".format(e))
+				l.unbind_s()
+				return None
+			try:
+				result = l.search_s(self.config['search_base'], \
+					ldap.SCOPE_SUBTREE, self.searchfilter(recipient), \
+					[self.config['public_folder_attribute']])
+			except ldap.LDAPError as e:
+				self.logger.logError("!--- LDAPError {}".format(e))
+			l.unbind_s()
+		return result
+
+	def publicfolder(self, recipient):
+		""" Check for ldap_public_folder_attribute_token and return folder.
+		Change type addition. """
+		destination_folder = []
+		result = self.searchquery(recipient)
+		if result:
+			tokenandfolder = (result[0][1]
+				.get(self.config['public_folder_attribute'])[0].decode('utf-8'))
+			if tokenandfolder:
+				destination_folder = tokenandfolder.split(':')[1]
+				if destination_folder:
+					self.logger.logDebug(("*--- Found public folder {}"
+						"for recipient {}".format(
+						destination_folder.encode('utf-8'),
+						recipient.encode('utf-8'))))
+		return destination_folder
+
+	def PreDelivery(self, session, addrbook, store, folder, message):
+		""" Original code from movetopublic.py with call to self.publicfolder().
+		Change type modification. """
+
+		props = message.GetProps([PR_RECEIVED_BY_EMAIL_ADDRESS_W], 0)
+		if props[0].ulPropTag != PR_RECEIVED_BY_EMAIL_ADDRESS_W:
+			self.logger.logError("!--- Not received by emailaddress")
+			return MP_CONTINUE,
+
+		recipient = props[0].Value.lower()
+		if not recipient:
+			self.logger.logError("!--- No recipient in props {}".format(props))
+			return MP_CONTINUE,
+
+		recipfolder = self.publicfolder(recipient)
+		if not recipfolder:
+			self.logger.logDebug(("*--- No public folder for recipient {}"
+				.format(recipient.encode('utf-8'))))
+			return MP_CONTINUE,
+
+		publicstore = GetPublicStore(session)
+		if not publicstore:
+			storeprops = store.GetProps([PR_MAILBOX_OWNER_ENTRYID], 0)
+			if storeprops[0].ulPropTag == PR_MAILBOX_OWNER_ENTRYID:
+				user = addrbook.OpenEntry(storeprops[0].Value, None, 0)
+				userprops = user.GetProps([PR_EC_COMPANY_NAME_W], 0)
+				if userprops[0].ulPropTag == PR_EC_COMPANY_NAME_W:
+					companyname = userprops[0].Value
+				else:
+					companyname = None
+
+				if not companyname:
+					self.logger.logError(("!--- Cannot open a public store."
+						' Use "kopano-storeadm -P"'
+						" to create one if it is missing."))
+					return MP_CONTINUE,
+
+				ema = store.QueryInterface(IID_IExchangeManageStore)
+				publicstoreid = ema.CreateStoreEntryID(None, companyname, MAPI_UNICODE)
+				publicstore = session.OpenMsgStore(0, publicstoreid, None, MDB_WRITE)
+
+		publicfolders = publicstore.OpenEntry(
+			publicstore.GetProps([PR_IPM_PUBLIC_FOLDERS_ENTRYID], 0)[0].Value,
+			None, MAPI_MODIFY)
+
+		folderlist = recipfolder.split('/')
+		folder = publicfolders
+		for foldername in folderlist:
+			if len(foldername) > 0:
+				if hexversion >= 0x03000000:
+					folder = folder.CreateFolder(0, foldername,
+						"Create by Move to Public plugin", None,
+						OPEN_IF_EXISTS | MAPI_UNICODE)
+				else:
+					folder = folder.CreateFolder(0, foldername,
+						"Create by Move to Public plugin", None, OPEN_IF_EXISTS)
+
+		msgnew = folder.CreateMessage(None, 0)
+		tags = message.GetPropList(MAPI_UNICODE)
+		message.CopyProps(tags, 0, None, IID_IMessage, msgnew, 0)
+
+		msgnew.SaveChanges(0)
+		folderid = folder.GetProps([PR_ENTRYID], 0)[0].Value
+		msgid = msgnew.GetProps([PR_ENTRYID], 0)[0].Value
+
+		publicstore.NotifyNewMail(NEWMAIL_NOTIFICATION(msgid, folderid, 0, None, 0))
+
+		self.logger.logInfo(("*--- Message moved to public folder {}"
+			.format(recipfolder)))
+
+		return MP_STOP_SUCCESS,

src/notused/plugin/ldaptocfg.py

@@ -0,0 +1,97 @@
+"""
+This code will query a LDAP/AD driectory server and updated the cgf file
+/etc/kopano/movetopublic.cfg
+/usr/share/kopano-dagent/python/plugins/movetopublic.cfg
+
+export PYTHONPATH=/usr/share/kopano-dagent/python
+
+"""
+from zconfig import ZConfigParser
+import configparser
+import ldap
+
+class KConfigParser(ZConfigParser):
+	""" allow !directive in cfg files """
+	def __init__(self, configfile, defaultconfig={}):
+		self.config = configparser.ConfigParser(defaults=defaultconfig, \
+			delimiters=('='), comment_prefixes=('#', '!'))
+		self.readZConfig(configfile)
+
+class ldapstores():
+	defaultconfig = {
+		'ldap_uri': None,
+		'ldap_search_base': None,
+		'ldap_bind_user': None,
+		'ldap_bind_passwd': None,
+		'ldap_user_unique_attribute': "uid",
+		'ldap_user_search_filter': "(kopanoAccount=1)",
+#		'ldap_user_search_filter': "(&(kopanoAccount=1)(kopanoResourceType=publicFolder:*))",
+		'ldap_public_folder_attribute': "kopanoResourceType",
+		'ldap_public_folder_attribute_token': "publicFolder"
+	}
+
+	def __init__(self, configfile = '/etc/kopano/ldap.cfg'):
+		self.readconfig(configfile)
+
+	def readconfig(self, configfile):
+		config = KConfigParser(configfile, self.defaultconfig)
+		options = [opt.split('_', 1)[1] for opt in self.defaultconfig.keys()]
+		self.config = config.getdict('ldap',options)
+		return self.config
+
+	def searchquery(self):
+		if (self.config['uri'] is None):
+			print ("ldap_uri is None")
+			sys.exit(0)
+		else:
+			l = ldap.initialize(self.config['uri'])
+			try:
+				l.protocol_version = ldap.VERSION3
+				l.simple_bind_s(self.config['bind_user'] or u'', \
+					self.config['bind_passwd'] or u'')
+			except ldap.INVALID_CREDENTIALS:
+				sys.exit(0)
+			except ldap.LDAPError as e:
+				print (e)
+				sys.exit(0)
+			try:
+				ldap_result_id = l.search(self.config['search_base'], \
+					ldap.SCOPE_SUBTREE, self.config['user_search_filter'], \
+					[self.config['user_unique_attribute'], \
+					self.config['public_folder_attribute']])
+				results = []
+				while 1:
+					result_type, result_data = l.result(ldap_result_id, 0)
+					if (result_data == []):
+						break
+					else:
+						if result_type == ldap.RES_SEARCH_ENTRY:
+							results.append(result_data[0])
+			except ldap.LDAPError as e:
+				print (e)
+			l.unbind_s()
+		return results
+
+	def findpublic(self):
+		stores = self.searchquery()
+		public = []
+		for store in stores:
+			recipient = store[1].get(self.config['user_unique_attribute'])
+			tokenandfolder = store[1].get(self.config['ldap_public_folder_attribute'])
+			if tokenandfolder:
+				token = tokenandfolder.split(':')[0]
+				destination_folder = tokenandfolder.split(':')[1]
+				if (token == self.config['ldap_public_folder_attribute_token']);
+					public[recipient] = destination_folder
+		return public
+
+	def printpublic(self, outputfile = '/etc/kopano/movetopublic.cfg'):
+		public = self.findpublic()
+		i = 1
+		for recipient in public.keys():
+			print ("rule%d_recipient = %s", i, recipient)
+			print ("rule%d_destination_folder = %s", i, public[recipient])
+			i += 1
+
+s = ldapstores()
+s.printpublic()

test/.gitignore

@@ -1,3 +0,0 @@
-local.*
-ssl
-acme

test/Makefile

@@ -1,37 +1,26 @@
+# Makefile
+#
+# test
+#
+
 -include    *.mk
 
-IMG_REPO ?= mlan/kopano
-IMG_VER  ?= latest
-_ver      = $(if $(findstring latest,$(1)),$(2),$(1)-$(2))
-_ip       = $(shell docker inspect -f \
-	'{{range .NetworkSettings.Networks}}{{println .IPAddress}}{{end}}' \
-	$(1) | head -n1)
-
-TST_NAME ?= test
-SSL_KEYF ?= priv_key.pem
-SSL_CRTF ?= cert.pem
-SSL_CRTD ?= 30
-SSL_ACMF ?= acme.json
-TST_SSLD ?= ssl
-TST_ACMD ?= acme
-TST_KEY  ?= $(TST_SSLD)/$(SSL_KEYF)
-TST_CERT ?= $(TST_SSLD)/$(SSL_CRTF)
-TST_ACME ?= $(TST_ACMD)/$(SSL_ACMF)
+TST_REPO ?= mlan/kopano
+TST_VER  ?= latest
+_ver      = $(if $(findstring latest,$(1)),$(2),$(2)-$(1))
 
 NET_NAME ?= test-net
 NET_ENV  ?= --network $(NET_NAME)
 
-LDAP_BAS ?= dc=example,dc=com
-LDAP_UOU ?= users
-LDAP_UOB ?= posixAccount
-LDAP_GOU ?= groups
-LDAP_FOU ?= "(&(objectclass=$(LDAP_UOB))(mail=%s))"
-LDAP_FPW ?= "(&(objectclass=$(LDAP_UOB))(uid=%u))"
-LDAP_APW ?= uid=user
+AD_BASE  ?= dc=example,dc=com
+AD_DOM   ?= $(call ad_sub_dot, $(AD_BASE))
+AD_DC    ?= $(call ad_cut_dot, 1, 1, $(AD_DOM))
+AD_GRP_OU ?= groups
+AD_USR_OB ?= kopano-user
+AD_USR_OU ?= users
+AD_USR_CN ?= hero
+AD_USR_PW ?= enigma
 
-MAIL_DOM ?= example.com
-MAIL_US1 ?= hero
-MAIL_PW1 ?= enigma
 MAIL_SUB ?= ~~~test~message~~~
 MAIL_MSG ?= Enjoy!
 
@@ -46,10 +35,10 @@ SQL_ENV  ?= \
 -e MYSQL_PASSWORD=$(SQL_PASS) \
 
 APP_NAME ?= app
-APP_IMG  ?=
-APP_FQDN ?= $(APP_NAME).$(MAIL_DOM)
+APP_FQDN ?= $(APP_NAME).$(AD_DOM)
+APP_CERT ?= ssl/$(APP_FQDN).crt
+APP_KEY  ?= ssl/$(APP_FQDN).key
 APP_VOL  ?=
-APP_SSLD ?= /etc/kopano/ssl
 APP_SLOG ?= 7
 APP_ALOG ?= 6
 APP_ENV  ?= $(NET_ENV) $(SQL_ENV) \
@@ -58,33 +47,36 @@ APP_ENV  ?= $(NET_ENV) $(SQL_ENV) \
 -e MYSQL_HOST=$(DB_NAME) \
 -e USER_PLUGIN=ldap \
 -e LDAP_URI=ldap://$(AUT_NAME):389/ \
--e LDAP_SEARCH_BASE=$(LDAP_BAS) \
--e LDAP_USER_TYPE_ATTRIBUTE_VALUE=$(LDAP_UOB) \
--e LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=$(LDAP_GOU) \
+-e LDAP_SEARCH_BASE=$(AD_BASE) \
+-e LDAP_USER_TYPE_ATTRIBUTE_VALUE=$(AD_USR_OB) \
+-e LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=$(AD_GRP_OU) \
+-e LDAP_PROPMAP= \
 -e IMAP_LISTEN=*:143 \
 -e POP3_LISTEN=*:110 \
 -e ICAL_LISTEN=*:8080 \
 -e DISABLED_FEATURES= \
+-e LOGLEVEL=LOGLEVEL_DEBUG \
 -e SYSLOG_LEVEL=$(APP_SLOG) \
 -e LOG_LEVEL=$(APP_ALOG)
 APPS_ENV ?= $(APP_ENV) \
 -e IMAPS_LISTEN=*:993 \
 -e POP3S_LISTEN=*:995 \
 -e ICALS_LISTEN=*:8443
-APPF_ENV ?= $(APPS_ENV) \
--v $(shell pwd)/$(TST_SSLD):/$(APP_SSLD)
 APPA_ENV ?= $(APPS_ENV) \
--v $(shell pwd)/$(TST_ACMD):/$(TST_ACMD)
-
-#-p "127.0.0.1:2003:2003"
+-v $(shell pwd)/acme:/acme
 
 AUT_NAME ?= auth
 AUT_IMG  ?= mlan/openldap
-AUT_FQDN ?= $(AUT_NAME).$(MAIL_DOM)
+AUT_FQDN ?= $(AUT_NAME).$(AD_DOM)
 AUT_VOL  ?=
 AUT_ENV  ?= $(NET_ENV) \
 --name $(AUT_NAME) \
---hostname $(AUT_FQDN) \
+--hostname $(AUT_FQDN)
+
+TAW_NAME ?= throwaway
+TAW_ENV  ?= --entrypoint /bin/bash \
+--name $(TAW_NAME) \
+--rm
 
 CURL_OPT ?= -s -v
 CURL_IMG ?= curlimages/curl
@@ -93,7 +85,7 @@ CURL_ENV ?= $(NET_ENV) \
 
 DB_NAME  ?= db
 DB_IMG   ?= mariadb
-DB_FQDN  ?= $(DB_NAME).$(MAIL_DOM)
+DB_FQDN  ?= $(DB_NAME).$(AD_DOM)
 DB_VOL   ?=
 DB_CMD   ?= --log_warnings=1
 DB_ENV   ?= $(NET_ENV) $(SQL_ENV) \
@@ -108,6 +100,33 @@ TST_W8S2 ?= 20
 TST_W8L1 ?= 20
 TST_W8L2 ?= 120
 
+export define LDIF_ADD_DATA
+dn: $(AD_BASE)
+objectClass: organization
+objectClass: dcObject
+dc: $(AD_DC)
+o: $(AD_DOM)
+
+dn: ou=$(AD_USR_OU),$(AD_BASE)
+objectClass: organizationalUnit
+ou: $(AD_USR_OU)
+
+dn: ou=$(AD_GRP_OU),$(AD_BASE)
+objectClass: organizationalUnit
+ou: $(AD_GRP_OU)
+
+dn: uid=$(AD_USR_CN),ou=$(AD_USR_OU),$(AD_BASE)
+objectClass: inetOrgPerson
+objectClass: $(AD_USR_OB)
+cn: $(AD_USR_CN)
+sn: $(AD_USR_CN)
+uid: $(AD_USR_CN)
+mail: $(AD_USR_CN)@$(AD_DOM)
+kopanoAccount: 1
+userPassword: $(AD_USR_PW)
+endef
+
+
 variables:
 	make -pn | grep -A1 "^# makefile"| grep -v "^#\|^--" | sort | uniq
 
@@ -119,10 +138,12 @@ test-all: test-up_0 test_1 test_2 test_3
 
 test_%: test-up_% test-waitl_% test-logs_% test-service_% test-down_%
 	
+
 test-pull:
 	$(addprefix docker pull ,$(CURL_IMG); $(AUT_IMG); $(DB_IMG))
 
 test-up_0: test-up-net
+	#
 	#
 	#
 	# test (0) run without envvars (is there smoke?)
@@ -130,23 +151,25 @@ test-up_0: test-up-net
 	# run containers see if there are logs and stop.
 	#
 	#
-	docker run -d $(APP_ENV) $(APP_VOL) $(IMG_REPO):$(call _ver,$(IMG_VER),core)
+	docker run -d --name $(APP_NAME) $(TST_REPO):$(call _ver,$(TST_VER),core)
 	sleep $(TST_W8L1)
 	docker container logs $(APP_NAME) | grep 'docker-entrypoint.sh'
 	docker rm -fv $(APP_NAME)
 	sleep $(TST_W8S1)
-	docker run -d $(APP_ENV) $(APP_VOL) $(IMG_REPO):$(call _ver,$(IMG_VER),full)
+	docker run -d --name $(APP_NAME) $(TST_REPO):$(call _ver,$(TST_VER),full)
 	sleep $(TST_W8L1)
 	docker container logs $(APP_NAME) | grep 'docker-entrypoint.sh'
 	docker rm -fv $(APP_NAME)
 	sleep $(TST_W8S1)
 	#
 	#
-	# test (0) successful
+	# test (0) success ☺
+	#
 	#
 	#
 
-test-up_1: test-up-net test-up-auth_1
+test-up_1: test-up-net test-up-deps_1 test-up-auth_1
+	#
 	#
 	#
 	# test (1) ldap auth, sql db, and mail send recv
@@ -156,11 +179,10 @@ test-up_1: test-up-net test-up-auth_1
 	# recv: curl imap://app
 	#
 	#
-	docker run -d $(DB_ENV) $(DB_VOL) $(DB_IMG) $(DB_CMD)
-	sleep $(TST_W8L1)
-	docker run -d $(APP_ENV) $(APP_VOL) $(IMG_REPO):$(call _ver,$(IMG_VER),core)
+	docker run -d $(APP_ENV) $(APP_VOL) $(TST_REPO):$(call _ver,$(TST_VER),core)
 
-test-up_2: test-up-net test-up-auth_2 $(TST_CERT)
+test-up_2: test-up-net test-up-deps_2 test-up-auth_2
+	#
 	#
 	#
 	# test (2) ldap auth, sql db, mail send recv secure using pem cert files
@@ -170,11 +192,10 @@ test-up_2: test-up-net test-up-auth_2 $(TST_CERT)
 	# recv: curl imaps://app
 	#
 	#
-	docker run -d $(DB_ENV) $(DB_VOL) $(DB_IMG) $(DB_CMD)
-	sleep $(TST_W8L1)
-	docker run -d $(APPF_ENV) $(APP_VOL) $(IMG_REPO):$(call _ver,$(IMG_VER),core)
+	docker run -d $(APPS_ENV) $(APP_VOL) $(TST_REPO):$(call _ver,$(TST_VER),core)
 
-test-up_3: test-up-net test-up-auth_3 $(TST_ACME)
+test-up_3: test-up-net test-up-deps_3 test-up-auth_3 acme/acme.json
+	#
 	#
 	#
 	# test (3) ldap auth, sql db, mail send recv secure using acme cert
@@ -185,9 +206,7 @@ test-up_3: test-up-net test-up-auth_3 $(TST_ACME)
 	# web: curl http://app
 	#
 	#
-	docker run -d $(DB_ENV) $(DB_VOL) $(DB_IMG) $(DB_CMD)
-	sleep $(TST_W8L1)
-	docker run -d $(APPA_ENV) $(APP_VOL) $(IMG_REPO):$(call _ver,$(IMG_VER),full)
+	docker run -d $(APPA_ENV) $(APP_VOL) $(TST_REPO):$(call _ver,$(TST_VER),full)
 
 test-up: test-up_1
 
@@ -196,7 +215,8 @@ test-service: test-service_0
 test-service_%: test-pop3_% test-imap_% test-http_%
 	#
 	#
-	# test ($*) successful
+	# test ($*) success ☺
+	#
 	#
 	#
 
@@ -216,31 +236,34 @@ test-up-net:
 	docker network create $(NET_NAME) 2>/dev/null || true
 
 test-down-net:
-	docker network rm $(NET_NAME) || true
-
-test-down: test-down_0
 	docker network rm $(NET_NAME) 2>/dev/null || true
 
+test-down: test-down_0 test-down-net acme-destroy
+	
+
 test-down_%:
-	docker rm -fv $(APP_NAME) $(DB_NAME) $(AUT_NAME) 2>/dev/null || true
-	if [ $* -ge 0 ]; then sleep $(TST_W8S1); fi
+	@docker rm -fv $(APP_NAME) $(DB_NAME) $(AUT_NAME) $(TAW_NAME)  2>/dev/null || true
+	@if [ $* -ge 0 ]; then sleep $(TST_W8S1); fi
+
+test-up-deps_%:
+	docker run -d $(DB_ENV) $(DB_VOL) $(DB_IMG) $(DB_CMD)
+	docker run -d $(AUT_ENV) $(AUT_VOL) $(AUT_IMG)
 
 test-up-auth_%:
-	docker run -d $(AUT_ENV) $(AUT_VOL) $(AUT_IMG)
 	sleep $(TST_W8L1)
-	printf "dn: ou=$(LDAP_UOU),$(LDAP_BAS)\nchangetype: add\nobjectClass: organizationalUnit\nobjectClass: top\nou: $(LDAP_UOU)\n\ndn: ou=$(LDAP_GOU),$(LDAP_BAS)\nchangetype: add\nobjectClass: organizationalUnit\nobjectClass: top\nou: $(LDAP_GOU)\n\ndn: uid=$(MAIL_US1),ou=$(LDAP_UOU),$(LDAP_BAS)\nchangetype: add\nobjectClass: top\nobjectClass: inetOrgPerson\nobjectClass: $(LDAP_UOB)\ncn: $(MAIL_US1)\nsn: $(MAIL_US1)\nuid: $(MAIL_US1)\nmail: $(MAIL_US1)@$(MAIL_DOM)\nuidNumber: 1234\ngidNumber: 1234\nhomeDirectory: /home/$(MAIL_US1)\nuserPassword: $(MAIL_PW1)\n" \
-	| docker exec -i $(AUT_NAME) ldap modify
+	docker run $(TAW_ENV) $(TST_REPO):$(call _ver,$(TST_VER),core) \
+	zcat /usr/share/doc/kopano/kopano.ldif.gz \
+	| docker exec -i $(AUT_NAME) ldapadd -Q
+	echo "$$LDIF_ADD_DATA" | docker exec -i $(AUT_NAME) ldapadd -Q
 
 test-lmtp: test-lmtp_0
 test-lmtp_%:
-	printf "LHLO mx\nMAIL FROM: <test@example.biz>\nRCPT TO: <$(MAIL_US1)@$(MAIL_DOM)>\nDATA\nFrom: A tester <test@example.biz>\nTo: <$(MAIL_US1)@$(MAIL_DOM)>\nDate: $$(date)\nSubject: $(MAIL_SUB)$*\n$(MAIL_MSG)$*\n.\nQUIT\n"\
-	| nc -C $(call _ip,$(APP_NAME)) 2003 # > /dev/null
-#	| nc -C localhost 2003 # > /dev/null
+	printf "LHLO mx\nMAIL FROM: <test@example.biz>\nRCPT TO: <$(AD_USR_CN)@$(AD_DOM)>\nDATA\nFrom: A tester <test@example.biz>\nTo: <$(AD_USR_CN)@$(AD_DOM)>\nDate: $$(date)\nSubject: $(MAIL_SUB)$*\n$(MAIL_MSG)$*\n.\nQUIT\n"\
+	| nc -C $(call dkr_cnt_ip,$(APP_NAME)) 2003 # > /dev/null
 	case $* in [1-3]) sleep $(TST_W8S1);; [4-9]) sleep $(TST_W8S2);; esac
 
 test-cfg_%:
 	$(eval cfg_s := $(shell [ $* -ge 2 ] && echo s))
-	$(eval cfg_s := $(shell [ $* -ge 2 ] && echo s))
 
 # Note: cannot use prereq also in a proper target since it will change the
 # prereq order, eg. avoid the "3" in test-imaps: test-imap_3
@@ -248,13 +271,13 @@ test-imap: test-imap_0
 test-imaps: test-imap_9
 test-imap_%: test-cfg_% test-lmtp_%
 	docker run $(CURL_ENV) $(CURL_IMG) $(CURL_OPT) imap$(cfg_s)://$(APP_NAME)/inbox \
-	--ssl --anyauth -k -X "fetch 1 all" -u $(MAIL_US1):$(MAIL_PW1) | grep $(GREP_ENV) $(MAIL_SUB)$*
+	--ssl --anyauth -k -X "fetch 1 all" -u $(AD_USR_CN):$(AD_USR_PW) | grep $(GREP_ENV) $(MAIL_SUB)$*
 
 test-pop3: test-pop3_0
 test-pop3s: test-pop3_9
 test-pop3_%: test-cfg_% test-lmtp_%
 	docker run $(CURL_ENV) $(CURL_IMG) $(CURL_OPT) pop3$(cfg_s)://$(APP_NAME)/1 \
-	--ssl --anyauth -k -u $(MAIL_US1):$(MAIL_PW1) | grep $(GREP_ENV) $(MAIL_SUB)$*
+	--ssl --anyauth -k -u $(AD_USR_CN):$(AD_USR_PW) | grep $(GREP_ENV) $(MAIL_SUB)$*
 
 test-http: test-http_9
 test-http_%: test-cfg_%
@@ -288,22 +311,8 @@ test-htop: test-debugtools
 test-tls: #--starttls imap
 	docker run --rm -it $(NET_ENV) drwetter/testssl.sh $(APP_NAME):993 || true
 
-test-ssl-gen: $(TST_ACME)
+acme-destroy: ssl-destroy
+	rm -f acme/*
 
-test-ssl-rm:
-	rm -rf $(TST_SSLD) $(TST_ACMD)
-#	rm $(TST_KEY) $(TST_CRT) $(TST_ACME)
-
-$(TST_ACME): $(TST_ACMD) $(TST_CERT)
-	bin/gen-acme-json.sh $(MAIL_US1)@$(MAIL_DOM) $(APP_FQDN) $(TST_KEY) $(TST_CERT) > $(TST_ACME)
-
-$(TST_CERT): $(TST_KEY)
-	openssl req -x509 -utf8 -new -batch -days $(SSL_CRTD) \
-	-subj "/CN=$(APP_FQDN)" -key $(TST_KEY) -out $@
-
-$(TST_KEY): $(TST_SSLD)
-	openssl genrsa -out $@
-	chmod a+r $@
-
-$(TST_SSLD) $(TST_ACMD):
-	mkdir -p $@
+acme/acme.json: $(APP_CERT)
+	bin/gen-acme-json.sh $(AD_USR_CN)@$(AD_DOM) $(APP_FQDN) $(APP_KEY) $(APP_CERT) > $@

test/acme/.gitignore

@@ -0,0 +1,4 @@
+# Ignore everything in this directory
+*
+# Except this file
+!.gitignore

test/ad.mk

@@ -0,0 +1,38 @@
+# ad.mk
+#
+# AD and LDAP   make-functions
+#
+
+#
+# chars
+#
+char_null  :=
+char_space := $(char_null) #
+char_comma := ,
+char_dot   := .
+char_colon := :
+
+#
+# $(call ad_sub_dc,example.com) -> dc=example,dc=com
+#
+ad_sub_dc   = $(subst $(char_space),$(char_comma),$(addprefix dc=, $(subst ., ,$(1))))
+#
+# $(call ad_sub_dot,dc=example,dc=com) -> example.com
+#
+ad_sub_dot  = $(subst $(char_comma)dc=,$(char_dot),$(patsubst dc=%,%,$(1)))
+#
+# $(call ad_cat_dn,admin,dc=example,dc=com) -> cn=admin,dc=example,dc=com
+#
+ad_cat_dn   = cn=$(1),$(2)
+#
+# $(call ad_cut_dot,1,1,example.com) -> example
+#
+ad_cut_dot  = $(subst $(char_space),$(char_dot),$(wordlist $(1), $(2), $(subst $(char_dot),$(char_space),$(3))))
+#
+# $(call ad_rootdc,2,9,adm.dom.org:secret) -> dom.org
+#
+ad_rootdc   = $(subst $(char_space),$(char_dot),$(wordlist $(1), $(2), $(subst $(char_dot),$(char_space),$(firstword $(subst $(char_colon),$(char_space),$(3))))))
+#
+# $(call ad_rootpw,adm.dom.org:secret) -> secret
+#
+ad_rootpw   = $(lastword $(subst $(char_colon),$(char_space),$(1)))

test/bin/gen-acme-json.sh

@@ -5,6 +5,13 @@ host=$2
 keyfile=$3
 certfile=$4
 
+#
+# The "PrivateKey": attribute needs a PKCS#1 key without tags and line breaks
+# "openssl req -newkey rsa" generates a key stored in PKCS#8 so needs conversion
+#
+#acme_strip_tag() { openssl rsa -in $1 | sed '/^-----/d' | sed ':a;N;$!ba;s/\n//g' ;}
+acme_strip_tag() { sed '/^-----/d' $1 | sed ':a;N;$!ba;s/\n//g' ;}
+
 cat <<-!cat
 {
   "Account": {
@@ -18,7 +25,7 @@ cat <<-!cat
       },
       "uri": "https://acme-v02.api.letsencrypt.org/acme/acct/$RANDOM"
     },
-    "PrivateKey": "$(sed '/^-----/d' $keyfile | sed ':a;N;$!ba;s/\n//g')",
+    "PrivateKey": "$(acme_strip_tag $keyfile)",
     "KeyType": "2048"
   },
   "Certificates": [

test/dkr.mk

@@ -0,0 +1,34 @@
+# dkr.mk
+#
+# Container   make-functions
+#
+
+#
+# $(call dkr_srv_cnt,app) -> d03dda046e0b90c...
+#
+dkr_srv_cnt = $(shell docker-compose ps -q $(1) | head -n1)
+#
+# $(call dkr_cnt_ip,demo_app_1) -> 172.28.0.3
+#
+dkr_cnt_ip   = $(shell docker inspect -f \
+	'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' \
+	$(1) | head -n1)
+#
+# $(call dkr_srv_ip,app) -> 172.28.0.3
+#
+dkr_srv_ip   = $(shell docker inspect -f \
+	'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' \
+	$$(docker-compose ps -q $(1)) | head -n1)
+#
+#cnt_ip_old = $(shell docker inspect -f \
+#	'{{range .NetworkSettings.Networks}}{{println .IPAddress}}{{end}}' \
+#	$(1) | head -n1)
+
+#
+# List IPs of containers
+#
+ip-list:
+	@for srv in $$(docker ps --format "{{.Names}}"); do \
+	echo $$srv $$(docker inspect -f \
+	'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $$srv); \
+	done | column -t

test/ssl.mk

@@ -0,0 +1,125 @@
+# ssl.mk
+#
+# SSL and TLS   make-functions
+#
+
+SSL_O    ?= example.com
+SSL_KEY  ?= rsa:2048 # rsa:2048 rsa:4096
+SSL_MAIL ?=
+SSL_PASS ?= secret
+SSL_SAN  ?=
+SSL_TRST ?=
+
+#
+# Usage: OpenLDAP
+#
+#SSL_O     = $(AD_DOM)
+#target: ssl/auth.crt ssl/demo.crt
+
+#
+# Usage: SMIME
+#
+#SSL_O     = $(MAIL_DOMAIN)
+#SSL_MAIL  = auto
+#SSL_PASS  = $(AD_USR_PW)
+##SSL_TRST  = $(SSL_SMIME)
+#target: ssl/$(AD_USR_CN)@$(MAIL_DOMAIN).p12
+SSL_SMIME =  -setalias "Self Signed SMIME" -addtrust emailProtection \
+	-addreject clientAuth -addreject serverAuth
+
+#
+# Usage: SUbject Alternate Name SAN
+#
+#SSL_O     = example.com
+#SSL_SAN   = "subjectAltName=DNS:auth,DNS:*.docker"
+#target: ssl/auth.crt
+
+
+#
+# $(call ssl_subj,root,example.com,) -> -subj "/CN=root/O=example.com"
+# $(call ssl_subj,root,example.com,auto) -> -subj "/CN=root/O=example.com/emailAddress=root@example.com"
+# $(call ssl_subj,root,example.com,admin@my.org) -> -subj "/CN=root/O=example.com/emailAddress=admin@my.org"
+#
+ssl_subj  = -subj "/CN=$(1)/O=$(2)$(if $(3),/emailAddress=$(if $(findstring @,$(3)),$(3),$(1)@$(2)),)"
+
+#
+# $(call ssl_extfile,"subjectAltName=DNS:auth") -> -extfile <(printf "subjectAltName=DNS:auth")
+#
+ssl_extfile = $(if $(1),-extfile <(printf $(1)),)
+
+
+.PRECIOUS: %.crt %.csr %.key
+SHELL   = /bin/bash
+
+#
+# Personal information exchange file PKCS#12
+#
+%.p12: %.crt
+	openssl pkcs12 -export -in $< -inkey $*.key -out $@ \
+	-passout pass:$(SSL_PASS)
+
+#
+# Certificate PEM
+#
+%.crt: %.csr ssl/ca.crt
+	openssl x509 -req -in $< -CA $(@D)/ca.crt -CAkey $(@D)/ca.key -out $@ \
+	$(call ssl_extfile,$(SSL_SAN)) $(SSL_TRST) -CAcreateserial
+
+#
+# Certificate signing request PEM
+#
+%.csr: ssl
+	openssl req -new -newkey $(SSL_KEY) -nodes -keyout $*.key -out $@ \
+	$(call ssl_subj,$(*F),$(SSL_O),$(SSL_MAIL))
+
+#
+# Certificate authority certificate PEM
+#
+ssl/ca.crt: ssl
+	openssl req -x509 -new -newkey $(SSL_KEY) -nodes -keyout ssl/ca.key -out $@ \
+	$(call ssl_subj,root,$(SSL_O),$(SSL_MAIL))
+
+#
+# SSL directory
+#
+ssl:
+	mkdir -p $@
+
+#
+# Remove all files in SSL directory
+#
+ssl-destroy:
+	rm -f ssl/*
+
+#
+# Inspect all files in SSL directory
+#
+ssl-list:
+	@for file in $$(ls ssl/*); do \
+	case $$file in \
+	*.crt) \
+	printf "\e[33;1m%s\e[0m\n" $$file; \
+	openssl x509 -noout -issuer -subject -ext basicConstraints,keyUsage,extendedKeyUsage,subjectAltName -in $$file;; \
+	*.csr) \
+	printf "\e[33;1m%s\e[0m\n" $$file; \
+	openssl req -noout -subject -in $$file;; \
+	*.key) \
+	printf "\e[33;1m%s\e[0m\n" $$file; \
+	openssl rsa -text -noout -in $$file | head -n 1;; \
+	esac \
+	done
+
+ssl-inspect:
+	@for file in $$(ls ssl/*); do \
+	case $$file in \
+	*.crt) \
+	printf "\e[33;1m%s\e[0m " $$file; \
+	openssl x509 -text -noout -certopt no_sigdump,no_pubkey -in $$file;; \
+	*.csr) \
+	printf "\e[33;1m%s\e[0m " $$file; \
+	openssl req -text -noout -reqopt no_sigdump,no_pubkey,ext_default -in $$file;; \
+	*.key) \
+	printf "\e[33;1m%s\e[0m " $$file; \
+	openssl rsa -text -noout -in $$file | head -n 1;; \
+	esac \
+	done

test/ssl/.gitignore

@@ -0,0 +1,4 @@
+# Ignore everything in this directory
+*
+# Except this file
+!.gitignore