- [docker](Dockerfile) Update Z-Push package URL.

- [kopano](src/kopano) Now use ubuntu:20.04 in `kopano-webaddr.sh`.
- [docker](ROADMAP.md) Update roadmap.
2021-06-26 12:52:24 +02:00 · 2021-05-05 17:05:48 +02:00 · 2021-04-26 09:24:39 +02:00 · 2021-01-29 11:47:53 +01:00 · 2021-01-28 21:53:29 +01:00 · 2021-01-20 14:28:35 +01:00
37 changed files with 2320 additions and 319 deletions
--- a/.travis.yml
+++ b/.travis.yml
@ -1,10 +1,9 @@
-sudo: required
-
-services:
-  - docker
-
-install:
-  - make build-all
-
+language: ruby
+os: linux
+dist: bionic
+services: docker
+install: make build-all
+before_script: sudo apt-get install -y ldap-utils openssl jq
 script:
-  - make version
+  - make test-pull
+  - make test-all
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -1,3 +1,70 @@
+# 1.3.1
+
+- [docker](Dockerfile) Update Z-Push package URL.
+- [kopano](src/kopano) Now use ubuntu:20.04 in `kopano-webaddr.sh`.
+- [docker](ROADMAP.md) Update roadmap.
+
+# 1.3.0
+
+- [docker](Dockerfile) Now use ubuntu:20.04 as base image.
+- [docker](Makefile) Don't set BLD_ARG in Makefile.
+- [kopano](src/kopano/plugin/movetopublicldap.py) Need explicitly to set `allow_no_value=True` due to update in `configparser.3.8.py`.
+- [demo](demo) Update to `mta-apk_list` target.
+
+# 1.2.9
+
+- [kopano](src/kopano) Bug in kopano-dagent out-of-office now fixed.
+- [demo](demo) Make demo less dependent on that curl and firefox beeing installed on the host.
+- [demo](demo) Don't expose any ports on the host, avoiding potential conflicts.
+
+# 1.2.8
+
+- [test](test) Update to use `mlan/openldap:2`.
+- [demo](demo) Update to use `mlan/openldap:2`.
+
+# 1.2.7
+
+- [kopano](src/kopano/plugin/movetopublicldap.py) Add LDAP support to the [Move to public](https://documentation.kopano.io/kopanocore_administrator_manual/special_kc_configurations.html#move-to-public ) kopano-dagent python plugin.
+- [kopano](src/kopano/plugin/movetopublicldap.py) Add sample config file for the kopano-dagent python plugin movetopublicldap.
+
+# 1.2.6
+
+- [docker](src/docker/bin/docker-config.sh) Allow the crontab support to work also when the file `/etc/kopano/docker-crontab` is missing.
+
+# 1.2.5
+
+- [docker](Dockerfile) Install the [kopano archiver](https://documentation.kopano.io/kopano_archiver_manual/) in target `core`.
+- [docker](Dockerfile) Install the smime webapp plugin providing [S/MIME](https://kopano.com/blog/s-mime-plugin-description/).
+- [docker](Dockerfile) Install the mdm webapp plugin providing [Mobile Device Management](https://documentation.kopano.io/webapp_mdm_manual/).
+- [demo](demo) Add [S/MIME](https://kopano.com/blog/s-mime-plugin-description/) cert generation.
+- [kopano](src/kopano/entry.d/10-kopano-common) Fix bug in `kopano_apply_envvars_php()`.
+- [kopano](src/kopano/entry.d/10-kopano-common) Allow module based parameter names.
+- [docker](src/docker/bin/docker-config.sh) Add provision to set up crontab using envvars.
+- [docker](README.md) Updated documentation.
+- [kopano](src/kopano) `MIGRATE_CONFIG=2` make sure WebApp plugins have configuration files in place.
+- [docker](Dockerfile) Make a copy of the kopano-webapp config files.
+
+# 1.2.4
+
+- [kopano](src/kopano) Now, use man pages and sample config files to find valid keys to match envvar.
+- [kopano](src/kopano) Load Kopano LDAP attributes using `LDAP_PROPMAP=`.
+
+# 1.2.3
+
+- [kopano](src/kopano) Adding support for secure IMAPS, POP3S and ICALS.
+- [kopano](src/kopano) Now generate self-signed certs if needed.
+- [demo](demo) Enable IMAPS, POP3S, ICALS and SMTPS in demo.
+- [acme](src/acme) Added module providing Let's encrypt TLS certificates using ACME.
+- [acme](src/acme) Fixed ACME module shell (ash and bash) portability.
+- [test](test/Makefile) Arrange build tests.
+- [test](test/Makefile) Add SSL build tests.
+- [travis-ci](.travis.yml) Travis CI now run test-all.
+- [docker](src/docker/bin/docker-config.sh) Generalized `dc_cond_chown()`.
+- [kopano](src/kopano) Updated documentation and bug fix of support for secure IMAPS, POP3S and ICALS.
+- [kopano](src/kopano) New, `MIGRATE_CONFIG=all` tries to make configs compatible with new versions.
+- [test](test/Makefile) Some sleep time in between db and app speeds up overall test times.
+- [travis-ci](.travis.yml) Fixed test errors on Travis-CI, which was due to long SQL db initialization.
+
 # 1.2.2

 - [kopano](src/kopano) Adding support for gateway / IMAP iCAL search configuration via envvars in, `50-kopano-apply-envvars`.
@ -58,7 +125,7 @@
 - Added support of the environment variable `LMTP_LISTEN=*:2003`, due to misconfiguration of `kopano-dagent` in recent releases (8.7.84).
 - Simplified the health check.
 - Changed repository directory structure to a more general one.
- Renamed some build variables, e.g., `DOCKER_RUNSV_DIR` (was `docker_build_runit_root`).
+- Renamed some build variables, e.g., `DOCKER_RUNSV_DIR` (was `docker_build_runit_root`).
 - Cleaning up `Makefile`
 - Added more debug functionality in `demo/Makefile`

--- a/50
+++ b/50
@ -2,7 +2,7 @@
 # build arguments, amd64 is the default
 #
 ARG	DIST=ubuntu
-ARG	REL=18.04
+ARG	REL=20.04
 ARG	ARCH

 FROM	${ARCH:+$ARCH/}$DIST:$REL AS base
@ -13,11 +13,17 @@ ENV	DEBIAN_FRONTEND=noninteractive \
 	DOCKER_BIN_DIR=/usr/local/bin \
 	DOCKER_ENTRY_DIR=/etc/docker/entry.d \
 	DOCKER_EXIT_DIR=/etc/docker/exit.d \
+	DOCKER_CRONTAB_FILE=/etc/kopano/docker-crontab \
+	DOCKER_CRONTAB_DIR=/etc/cron.d \
 	DOCKER_CONF_DIR1=/etc/kopano \
+	DOCKER_SMPL_DIR1=/usr/share/doc/kopano/example-config \
+	DOCKER_PLUG_DIR=/usr/share/kopano-dagent/python/plugins \
 	DOCKER_CONF_DIR2=/usr/share/z-push \
 	DOCKER_APPL_LIB=/var/lib/kopano \
+	DOCKER_APPL_SSL_DIR=/etc/kopano/ssl \
+	DOCKER_ACME_SSL_DIR=/etc/ssl/acme \
 	KOPANO_SPAMD_LIB=/var/lib/kopano/spamd \
-	DOCKER_RUNAS=kopano \
+	DOCKER_APPL_RUNAS=kopano \
 	DOCKER_BUILD_DEB_DIR=/tmp/deb \
 	DOCKER_BUILD_PASSES=1 \
 	DOCKER_UNLOCK_FILE=/etc/kopano/.docker.unlock \
@ -29,6 +35,8 @@ ENV	DEBIAN_FRONTEND=noninteractive \
 COPY	src/*/bin $DOCKER_BIN_DIR/
 COPY	src/*/entry.d $DOCKER_ENTRY_DIR/
 COPY	src/*/exit.d $DOCKER_EXIT_DIR/
+COPY	src/*/config $DOCKER_CONF_DIR1/
+COPY	src/*/plugin $DOCKER_PLUG_DIR/

 #
 # Install helpers. Set bash as default shell. Setup syslogs service.
@ -42,7 +50,14 @@ RUN	apt-get update && apt-get install --yes --no-install-recommends \
 	ca-certificates \
 	tar \
 	gnupg \
-	&& docker-service.sh "syslogd -nO- -l$SYSLOG_LEVEL $SYSLOG_OPTIONS"
+	jq \
+	inotify-tools \
+	cron \
+	&& ln -s $DOCKER_CRONTAB_FILE $DOCKER_CRONTAB_DIR \
+	&& docker-service.sh \
+	"syslogd -nO- -l$SYSLOG_LEVEL $SYSLOG_OPTIONS" \
+	"cron -f"
+#	"cron -f -L 4"



@ -62,7 +77,7 @@ RUN	rm -f /etc/dpkg/dpkg.cfg.d/excludes \
 	man \
 	manpages \
 	bash-completion \
-	&& rm -r /var/lib/apt/lists/* 
+	&& rm -r /var/lib/apt/lists/*



@ -84,7 +99,6 @@ ENV	DEBIAN_FRONTEND=noninteractive \
 	DOCKER_BIN_DIR=/usr/local/bin \
 	LMTP_LISTEN=*:2003 \
 	SA_GROUP=kopano \
-	LOG_METHOD=syslog \
 	DOCKER_BUILD_DEB_DIR=/tmp/deb \
 	DOCKER_BUILD_PASSES=1
 #
@ -95,14 +109,24 @@ RUN	mkdir -p $DOCKER_BUILD_DEB_DIR \
 	https://download.kopano.io/community ${DIST} ${REL} ${ARCH}) \
 	&& echo "$webaddr<->${DIST} ${REL} ${ARCH}<-" \
 	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
+	&& webaddr=$(kopano-webaddr.sh archiver \
+	https://download.kopano.io/community ${DIST} ${REL} ${ARCH}) \
+	&& echo "$webaddr<->${DIST} ${REL} ${ARCH}<-" \
+	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
 	&& apt-get update \
 	&& for i in $(seq ${DOCKER_BUILD_PASSES}); do echo "\033[1;36mKOPANO CORE INSTALL PASS: $i\033[0m" \
 	&& dpkg --install --force-depends --skip-same-version --recursive $DOCKER_BUILD_DEB_DIR \
 	&& apt-get install --yes --no-install-recommends --fix-broken; done \
-	&& mkdir -p /var/lib/kopano/attachments && chown $DOCKER_RUNAS: /var/lib/kopano/attachments \
-	&& mkdir -p $KOPANO_SPAMD_LIB/ham && chown $DOCKER_RUNAS: $KOPANO_SPAMD_LIB/ham \
+	&& apt-get install --yes --no-install-recommends python3-ldap \
+	&& mkdir -p /var/lib/kopano/attachments && chown $DOCKER_APPL_RUNAS: /var/lib/kopano/attachments \
+	&& mkdir -p $DOCKER_APPL_SSL_DIR \
+	&& mkdir -p $DOCKER_ACME_SSL_DIR \
+	&& mkdir -p $KOPANO_SPAMD_LIB/ham && chown $DOCKER_APPL_RUNAS: $KOPANO_SPAMD_LIB/ham \
 	&& rm -rf $DOCKER_BUILD_DEB_DIR \
 	&& rm $DOCKER_CONF_DIR1/*.cfg \
+	&& . docker-common.sh \
+	&& . docker-config.sh \
+	&& dc_comment /etc/ssl/openssl.cnf RANDFILE \
 	&& docker-service.sh \
 	"kopano-dagent -l" \
 	"kopano-gateway" \
@ -156,6 +180,14 @@ RUN	apt-get install --yes --no-install-recommends apache2 libapache2-mod-php \
 	https://download.kopano.io/community ${DIST} ${REL} all) \
 	&& echo "$webaddr<->${DIST} ${REL} all<-" \
 	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
+	&& webaddr=$(kopano-webaddr.sh mdm \
+	https://download.kopano.io/community ${DIST} ${REL} all) \
+	&& echo "$webaddr<->${DIST} ${REL} all<-" \
+	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
+	&& webaddr=$(kopano-webaddr.sh smime \
+	https://download.kopano.io/community ${DIST} ${REL} ${ARCH}) \
+	&& echo "$webaddr<->${DIST} ${REL} all<-" \
+	&& curl $webaddr | tar -xzC $DOCKER_BUILD_DEB_DIR \
 	&& apt-get update \
 	&& for i in $(seq ${DOCKER_BUILD_PASSES}); do echo "\033[1;36mKOPANO WEBAPP INSTALL PASS: $i\033[0m" \
 	&& dpkg --install --force-depends --skip-same-version --recursive $DOCKER_BUILD_DEB_DIR \
@ -164,7 +196,6 @@ RUN	apt-get install --yes --no-install-recommends apache2 libapache2-mod-php \
 	&& . docker-common.sh \
 	&& . docker-config.sh \
 	&& dc_replace /etc/kopano/webapp/config.php 'define("SECURE_COOKIES", true);' 'define("SECURE_COOKIES", false);' \
-#	&& dc_fixmissing /etc/php/7.?/apache2/conf.d/kopano.ini /etc/php/7.?/mods-available/kopano.ini /etc/php5/conf.d/kopano.ini \
 	&& dc_replace /etc/apache2/sites-available/kopano-webapp.conf 'Alias /webapp /usr/share/kopano-webapp' '<VirtualHost *:80>\\nDocumentRoot /usr/share/kopano-webapp' \
 	&& echo '</VirtualHost>' >> /etc/apache2/sites-available/kopano-webapp.conf \
 	&& dc_modify /etc/apache2/apache2.conf '^ErrorLog' syslog:user \
@ -177,6 +208,7 @@ RUN	apt-get install --yes --no-install-recommends apache2 libapache2-mod-php \
 	&& a2dissite 000-default.conf \
 	&& a2ensite kopano-webapp \
 	&& rm -rf $DOCKER_BUILD_DEB_DIR \
+	&& cp -r $DOCKER_CONF_DIR1/webapp $DOCKER_SMPL_DIR1 \
 	&& docker-service.sh "-f -s /etc/apache2/envvars -q apache2 -DFOREGROUND -DNO_DETACH -k start"
 #
 # Ports
@ -204,7 +236,7 @@ ENV	DEBIAN_FRONTEND=noninteractive \
 #
 # Add Z-Push repository and install Z-Push configured to be used with Kopano and Apache
 #
-RUN	debaddr="$(kopano-webaddr.sh --deb final http://repo.z-hub.io/z-push: ${DIST} ${REL})" \
+RUN	debaddr="$(kopano-webaddr.sh --deb final https://download.kopano.io/zhub/z-push: ${DIST} ${REL})" \
 	&& echo "deb $debaddr/ /" > /etc/apt/sources.list.d/z-push.list \
 	&& wget -qO - $debaddr/Release.key | apt-key add - \
 	&& mkdir -p /var/lib/z-push && chown www-data: /var/lib/z-push \
--- a/45
+++ b/45
@ -1,21 +1,29 @@
+# Makefile
+#
+# build
+#
+
 -include    *.mk

-#BLD_ARG  ?= --build-arg DIST=ubuntu --build-arg REL=18.04 --build-arg ARCH=i386
-BLD_ARG  ?=
+#BLD_ARG  ?= --build-arg DIST=ubuntu --build-arg REL=20.04
 BLD_REPO ?= mlan/kopano
 BLD_VER  ?= latest
 BLD_TGT  ?= full
-SRC_VER  ?= $(shell src/kopano/bin/kopano-webaddr.sh -VV)

-_version  = $(if $(findstring $(BLD_TGT),$(1)),$(2),$(if $(findstring latest,$(2)),$(1),$(1)-$(2)))
+SRC_CMD  ?= src/kopano/bin/kopano-webaddr.sh -VV
+SRC_VER  ?= $(shell $(SRC_CMD))

-.PHONY:
+TST_REPO ?= $(BLD_REPO)
+TST_VER  ?= $(BLD_VER)
+TST_ENV  ?= -C test
+TST_TGTE ?= $(addprefix test-,all diff down env htop imap lmtp logs mail pop3 pull sh sv up)
+TST_TGTI ?= test_% test-up_%

-variables:
-	make -pn | grep -A1 "^# makefile"| grep -v "^#\|^--" | sort | uniq
+export TST_REPO TST_VER

-ps:
-	docker ps -a
+_version  = $(if $(findstring $(BLD_TGT),$(1)),\
+$(if $(findstring latest,$(2)),latest $(1) $(SRC_VER) $(1)-$(SRC_VER),$(2) $(1)-$(2)),\
+$(if $(findstring latest,$(2)),$(1) $(1)-$(SRC_VER),$(1)-$(2)))

 build-all: build_core build_full

@ -23,14 +31,25 @@ build: build_$(BLD_TGT)

 build_%: Dockerfile
 	docker build $(BLD_ARG) --target $* \
-	-t $(BLD_REPO):$(call _version,$*,$(BLD_VER)) \
-	-t $(BLD_REPO):$(call _version,$*,$(SRC_VER)) .
+	$(addprefix --tag $(BLD_REPO):,$(call _version,$*,$(BLD_VER))) .

 version:
-	src/kopano/bin/kopano-webaddr.sh -VV
+	$(SRC_CMD)
+
+variables:
+	make -pn | grep -A1 "^# makefile"| grep -v "^#\|^--" | sort | uniq
+
+ps:
+	docker ps -a

 prune:
-	docker image prune
+	docker image prune -f

 clean:
 	docker images | grep $(BLD_REPO) | awk '{print $$1 ":" $$2}' | uniq | xargs docker rmi
+
+$(TST_TGTE):
+	${MAKE} $(TST_ENV) $@
+
+$(TST_TGTI):
+	${MAKE} $(TST_ENV) $@
--- a/README.md
+++ b/README.md
@ -3,21 +3,27 @@
 ![travis-ci test](https://img.shields.io/travis/mlan/docker-kopano.svg?label=build&style=popout-square&logo=travis)
 ![docker build](https://img.shields.io/docker/cloud/build/mlan/kopano.svg?label=build&style=popout-square&logo=docker)
 ![image Size](https://img.shields.io/docker/image-size/mlan/kopano.svg?label=size&style=popout-square&logo=docker)
-![docker stars](https://img.shields.io/docker/stars/mlan/kopano.svg?label=stars&style=popout-square&logo=docker)
 ![docker pulls](https://img.shields.io/docker/pulls/mlan/kopano.svg?label=pulls&style=popout-square&logo=docker)
+![docker stars](https://img.shields.io/docker/stars/mlan/kopano.svg?label=stars&style=popout-square&logo=docker)
+![github stars](https://img.shields.io/github/stars/mlan/docker-kopano.svg?label=stars&style=popout-square&logo=github)

-This (non official) repository provides dockerized web mail service as well as Exchange ActiveSync (EAS), ICAL, IMAP and POP3 service. It is based on [Kopano](https://kopano.com) core components, as well as the Kopano WebApp and [Z-Push](http://z-push.org/). The image uses [nightly built packages](https://download.kopano.io/community/) which are provided by the Kopano community.
+This (non official) repository provides dockerized web mail service as well as Exchange ActiveSync (EAS), IMAP, POP3 and ICAL service (and their secure variants IMAPS, POP3S and ICALS). It is based on [Kopano](https://kopano.com) core components, as well as the Kopano WebApp and [Z-Push](http://z-push.org/). The image uses [nightly built packages](https://download.kopano.io/community/) which are provided by the Kopano community.

 Hopefully this repository can be retired once the Kopano community make official images available. To learn more about this activity see [zokradonh/kopano-docker](https://github.com/zokradonh/kopano-docker).

 ## Features

- Groupware server [Kopano WebApp](https://kopano.io/)
+- [Kopano WebApp](https://kopano.io/) the main client to access all the features provided by Kopano core
 - [Exchange ActiveSync (EAS)](https://en.wikipedia.org/wiki/Exchange_ActiveSync) server [Z-Push](http://z-push.org/)
- Multi-staged build providing the images `full`, `debugtools` and `core`
+- IMAP, POP3 and ICAL services provided by Kopano core
+- Secure protocols IMAPS, POP3S and ICALS
+- Hooks for integrating [Let’s Encrypt](https://letsencrypt.org/) LTS certificates using the reverse proxy [Traefik](https://docs.traefik.io/)
+- Multi-staged build providing the images `full` and `core`
 - Configuration using environment variables
 - Log directed to docker daemon with configurable level
- Built in utility script [conf](src/docker/bin/conf) helping configuring Kopano components, WebApp and Z-Push
+- Built in utility script [run](src/docker/bin/run) helping configuring Kopano components, WebApp and Z-Push
+- [Move to public with LDAP lookup](#move-to-public-with-ldap-lookup)
+- [Crontab](https://en.wikipedia.org/wiki/Cron) support.
 - Health check
 - Hook for theming
 - Demo based on `docker-compose.yml` and `Makefile` files
@ -61,11 +67,14 @@ services:
    image: mlan/kopano
    networks:
      - backend
-    ports:
-      - "127.0.0.1:8008:80"    # WebApp & EAS (alt. HTTP)
-      - "127.0.0.1:110:110"    # POP3 (not needed if all devices can use EAS)
-      - "127.0.0.1:143:143"    # IMAP (not needed if all devices can use EAS)
-      - "127.0.0.1:8080:8080"  # CalDAV (not needed if all devices can use EAS)
+    ports:           # Expose ports to host interfaces
+      - "80:80"      # WebApp & EAS (alt. HTTP)
+      - "143:143"    # IMAP (not needed if all devices can use EAS)
+      - "110:110"    # POP3 (not needed if all devices can use EAS)
+      - "8080:8080"  # ICAL (not needed if all devices can use EAS)
+      - "993:993"    # IMAPS (not needed if all devices can use EAS)
+      - "995:995"    # POP3S (not needed if all devices can use EAS)
+      - "8443:8443"  # ICALS (not needed if all devices can use EAS)
    depends_on:
      - auth
      - db
@ -75,17 +84,24 @@ services:
      - LDAP_URI=ldap://auth:389/
      - MYSQL_HOST=db
      - SMTP_SERVER=mta
-      - LDAP_SEARCH_BASE=${LDAP_BASE-dc=example,dc=com}
-      - LDAP_USER_TYPE_ATTRIBUTE_VALUE=${LDAP_USEROBJ-posixAccount}
-      - LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=${LDAP_GROUPOBJ-posixGroup}
+      - LDAP_SEARCH_BASE=${AD_BASE-dc=example,dc=com}
+      - LDAP_USER_TYPE_ATTRIBUTE_VALUE=${AD_USR_OB-kopano-user}
+      - LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=${AD_GRP_OB-kopano-group}
+      - LDAP_GROUPMEMBERS_ATTRIBUTE_TYPE=dn
+      - LDAP_PROPMAP=
+      - DAGENT_PLUGINS=movetopublicldap
      - MYSQL_DATABASE=${MYSQL_DATABASE-kopano}
      - MYSQL_USER=${MYSQL_USER-kopano}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD-secret}
-      - POP3_LISTEN=*:110                       # also listen to eth0
      - IMAP_LISTEN=*:143                       # also listen to eth0
+      - POP3_LISTEN=*:110                       # also listen to eth0
      - ICAL_LISTEN=*:8080                      # also listen to eth0
-      - DISABLED_FEATURES=${DISABLED_FEATURES-} # also enable IMAP and POP3
+      - IMAPS_LISTEN=*:993                      # enable TLS
+      - POP3S_LISTEN=*:995                      # enable TLS
+      - ICALS_LISTEN=*:8443                     # enable TLS
+      - PLUGIN_SMIME_USER_DEFAULT_ENABLE_SMIME=true
      - SYSLOG_LEVEL=${SYSLOG_LEVEL-3}
+      - LOG_LEVEL=${LOG_LEVEL-3}
    volumes:
      - app-conf:/etc/kopano
      - app-atch:/var/lib/kopano/attachments
@ -100,15 +116,16 @@ services:
    hostname: ${MAIL_SRV-mx}.${MAIL_DOMAIN-example.com}
    networks:
      - backend
-    ports:
-      - "127.0.0.1:25:25"      # SMTP
+    ports:           # Expose ports to host interfaces
+      - "25:25"      # SMTP
+      - "465:465"    # SMTPS authentication required
    depends_on:
      - auth
    environment: # Virgin config, ignored on restarts unless FORCE_CONFIG given.
      - LDAP_HOST=auth
      - VIRTUAL_TRANSPORT=lmtp:app:2003
-      - LDAP_USER_BASE=ou=${LDAP_USEROU-users},${LDAP_BASE-dc=example,dc=com}
-      - LDAP_QUERY_FILTER_USER=(&(objectclass=${LDAP_USEROBJ-posixAccount})(mail=%s))
+      - LDAP_USER_BASE=ou=${AD_USR_OU-users},${AD_BASE-dc=example,dc=com}
+      - LDAP_QUERY_FILTER_USER=(&(objectclass=${AD_USR_OB-kopano-user})(mail=%s))
    volumes:
      - mta:/srv
      - app-spam:/var/lib/kopano/spamd          # kopano-spamd integration
@ -135,8 +152,10 @@ services:
    image: mlan/openldap
    networks:
      - backend
+    command: --root-cn ${AD_ROOT_CN-admin} --root-pw ${AD_ROOT_PW-secret}
    environment:
-      - LDAP_LOGLEVEL=parse
+      - LDAPBASE=${AD_BASE-dc=example,dc=com}
+      - LDAPDEBUG=${AD_DEBUG-parse}
    volumes:
      - auth:/srv
      - /etc/localtime:/etc/localtime:ro        # Use host timezone
@ -156,7 +175,7 @@ volumes:

 ## Demo

-This repository contains a [demo](demo) directory which hold the [docker-compose.yml](demo/docker-compose.yml) file as well as a [Makefile](demo/Makefile) which might come handy. Start with cloning the [github](https://github.com/mlan/docker-kopano) repository.
+This repository contains a [demo](demo) directory which hold the [docker-compose.yml](demo/docker-compose.yml) file as well as a [Makefile](demo/Makefile) which might come handy. To run the demo you need [docker-compose](https://docs.docker.com/compose/install/) installed. By default `curl` and `firefox` is expected to be installed, but if not, run `make utils-container` within the [demo](demo) directory once the repository has been cloned. The `make` utility works nicely with [bash-completion](https://github.com/scop/bash-completion) so it can be worth considering having it installed too. Once the dependencies are met, start with cloning the [github](https://github.com/mlan/docker-kopano) repository.

 ```bash
 git clone https://github.com/mlan/docker-kopano.git
@ -168,7 +187,7 @@ From within the [demo](demo) directory you can start the containers by typing:
 make init
 ```

-Then you can assess WebApp on the URL [`http://localhost:8008`](http://localhost:8008) and log in with the user name `demo` and password `demo` . You can send yourself a test email by typing:
+Now you can assess WebApp on the custom docker network at URL `http://app` and log in with the user name `demo` and password `demo`.

 ```bash
 make web
@ -180,7 +199,7 @@ You can send yourself a test email by typing:
 make test
 ```

-When you are done testing you can destroy the test containers by typing
+When you are done testing you can destroy the test containers and their volumes by typing:

 ```bash
 make destroy
@ -203,17 +222,22 @@ The unlock file approach was selected since it is difficult to accidentally _cre

 In the rare event that want to modify the configuration of an existing container you can override the default behavior by setting `FORCE_CONFIG=overwrite` to a no-empty string.

-## Environment variables
+## Environment variables and service parameters

-When you create the `mlan/kopano` container, you can adjust the configuration of the Kopano server by passing one or more environment variables or on the docker run command line. Note that any pre-existing configuration files within the container will be left untouched.
+When you create the `mlan/kopano` container, you can adjust the configuration of the Kopano server by defining one or more environment variables. During container initiation environment variables are matched against all possible parameters for all Kopano services. When matched, configuration files are updated with the value of the matching environment variable.

-To see all available configuration variables you can run `man` within the container by for example using the [Makefile](demo/Makefile) described above:
+To see all available configuration parameters you can run `run list_parms <service>` within the container or when when using the [demo](#demo) described above just type:

 ```bash
-make mail-app-man_server
+make app-parms_dagent
+make app-parms_server
 ```

-If you do, you will notice that configuration variable names are all lower case, but they will be matched with all uppercase environment variables by the container `docker-entrypoint.sh` script.
+### Overlapping parameter names
+
+Some services use the same parameter names. When such a parameter is set using en environment variable all configuration files of the related services will be updated. This is not always desired. To address this you can prefix the parameter name with the name of the service you which to target by using the following syntax: `<service>_<parameter>`.
+
+For example when using the [Kopano-archiver](https://documentation.kopano.io/kopano_archiver_manual/) service, you often want to use one SQL database for the server and one separate one for the archiver. In this situation you can have two separate SQL database containers, one at `db-srv` and the other at `db-arc`. To set the [`MYSQL_HOST`](#mysql_host) parameter in the two relevant configuration files use `SERVER_MYSQL_HOST=db-srv` and `ARCHIVER_MYSQL_HOST=db-arc`. If for some reason you want both services to call the same container, instead use: `MYSQL_HOST=db`.

 ## SQL database configuration

@ -251,27 +275,25 @@ When the `ATTACHMENT_STORAGE` option is `ATTACHMENT_STORAGE=files`, this option

 When creating the SQL container you can use environment variables to initiate it. For example, `MYSQL_ROOT_PASSWORD=topsecret`, `MYSQL_DATABASE=kopano`, `MYSQL_USER=kopano` and `MYSQL_PASSWORD=verysecret`.

-## User authentication `USER_PLUGIN`
+## User management `USER_PLUGIN`

-Kopano supports three different ways to manage user authentication. Use the `USER_PLUGIN` environment variable to select the source of the user base. Possible values are: `db` (default), `ldap` and `unix`.
+Kopano supports three different plugins for user management. Use the `USER_PLUGIN` environment variable to select the source of the user base. Possible values are: `db` (default), `ldap` and `unix`.

- `db`: Retrieve the users from the Kopano database. Use the kopano-admin tool to create users and groups. There are no additional settings for this plug-in.
+`db`: Retrieve the users from the Kopano database. Use the `kopano-admin` tool to create users and groups. There are no additional settings for this plug-in.

-`ldap`: Retrieve the users and groups information from an LDAP server. All additional LDAP settings are needed see below
+`ldap`: Retrieve the users and groups information from an LDAP directory server. Additional LDAP settings are needed, see below.

 `unix`: Retrieve the users and groups information from the Linux password files. This option is probably not interesting here.

-### LDAP authentication
+### Accessing an LDAP directory server

-An LDAP server with user accounts configured to be used with Kopano is needed, but how to set one up is out of our scope here, instead see: [Kopano Knowledge Base/Install and optimize OpenLDAP for use with Kopano Groupware Core](https://kb.kopano.io/display/WIKI/Install+and+optimize+OpenLDAP+for+use+with+Kopano+Groupware+Core).
+The `USER_PLUGIN=ldap` retrieves user information from an LDAP directory server. A brief description of how that is achieved is described in [Setup an LDAP directory server](#setup-an-ldap-directory-server). Once the LDAP directory server is up and running, the `mlan/kopano` container can be configured to use it using environment variables.

-Once the LDAP server is up and running, the `mlan/kopano` container can be configured to use it using environment variables. In addition to the variables discussed below also set `USER_PLUGIN=ldap`.
-
-#### `LDAP_URI`
+#### Host address `LDAP_URI`

 Specifies the URI of one or more LDAP server(s) to use, without any DN portion, such as `ldap://server:389/`, `ldaps://server:636/` or `ldapi:///`. Defaults: `LDAP_URI=ldap://localhost:389/`.

-The historic directives `LDAP_HOST`, `LDAP_PORT`, `LDAP_PROTOCOL` are no longer supported (8.7.85).
+Note that. the historic directives `LDAP_HOST`, `LDAP_PORT`, `LDAP_PROTOCOL` are no longer supported (8.7.85).

 #### `LDAP_SEARCH_BASE`

@ -289,19 +311,89 @@ This variable determines what defines a valid Kopano group. Default: `LDAP_GROUP

 Adds an extra filter to the user search. Default `LDAP_USER_SEARCH_FILTER=`

-Hint: Use the `kopanoAccount` attribute in the filter to differentiate between non-kopano and kopano users.
+Hint: Use the `kopanoAccount` attribute in the filter to differentiate between non-Kopano and Kopano users.

-### Enabling IMAP, POP3 and ICAL
+#### `LDAP_BIND_USER`, `LDAP_BIND_PASSWD`

-By default the IMAP and POP3 services are disabled for all users. Set the environment variable `DISABLED_FEATURES=` to an empty string to enable both IMAP and POP3 for all users.
+The defaults for these environment variables are empty. If you cannot bind anonymously, do it with this distinguished name and password. Example: LDAP_BIND_USER=cn=admin,dc=example,dc=com, LDAP_BIND_PASSWD=secret.
+
+### Kopano LDAP attributes `LDAP_PROPMAP`
+
+The Kopano services needs to know which of the users LDAP attributes, like addresses, phone numbers and company information, to use. This information is defined in the `propmap` file, which is included in the Kopano installation files here `/usr/share/kopano/ldap.propmap.cfg`. When using `USER_PLUGIN=ldap` this LDAP `propmap` file is used by the Kopano services by setting `LDAP_PROPMAP=` to an empty string. Optionally you can use another file, for example`LDAP_PROPMAP=/etc/kopano/ldap.propmap.cfg`. If no file can be found there the installed one will be copied there.
+
+## Enabling IMAP, POP3 and ICAL
+
+By default the [IMAP](https://www.atmail.com/blog/imap-commands/) and POP3 services are disabled for all users. Set the environment variable `DISABLED_FEATURES=` to an empty string to enable both IMAP and POP3 for all users. You can override this setting for each user independently by enabling or disabling features in the LDAP directory server see, [Setup an LDAP directory server](#setup-an-ldap-directory-server).

 #### `DISABLED_FEATURES`

 The environment variable `DISABLED_FEATURES` take a space separated list of features. Currently it may contain the following features: `imap`, `mobile`, `outlook`, `pop3` and `webapp`. Default: `DISABLED_FEATURES="imap pop3"`

-#### `POP3_LISTEN`, `IMAP_LISTEN` and `ICAL_LISTEN`
+#### `IMAP_LISTEN`, `POP3_LISTEN`and `ICAL_LISTEN`

-By default the kopano-gateway and kopano-ical services are configured to only listen on the loopback interface. To be able to access these services we need them to listen to any interface. This is achieved by setting `POP3_LISTEN=*:110`, `IMAP_LISTEN=*:143` and `ICAL_LISTEN=*:8080`. These port numbers can be changed if desired.
+By default the kopano-gateway and kopano-ical services are configured to only listen on the loop-back interface. To be able to access these services we need them to listen to any interface. This is achieved by setting `IMAP_LISTEN=*:143`, `POP3_LISTEN=*:110` and `ICAL_LISTEN=*:8080`. These port numbers can be changed if desired.
+
+## Enabling IMAPS, POP3S and ICALS
+
+By default the secure protocols are not enabled.
+
+#### `IMAPS_LISTEN`, `POP3S_LISTEN`and `ICALS_LISTEN`
+
+To enable secure access we need to explicitly define their listening ports. This is achieved by setting any combination of `IMAPS_LISTEN=*:993`, `POP3S_LISTEN=*:995` and `ICALS_LISTEN=*:8443`. These port numbers can be changed if desired.
+
+If any of `IMAPS_LISTEN`, `POP3S_LISTEN` and `ICALS_LISTEN` are explicitly defined but there are no certificate files defined, a self-signed certificate will be generated when the container is created.
+
+### SSL/LTS certificate and private key
+
+For most deployments a trusted SSL/TLS certificate is desired. During startup the `mlan/kopano` looks for [RSA](https://en.wikipedia.org/wiki/RSA_(cryptosystem)) [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) certificate and private key with these specific names: `/etc/kopano/ssl/cert.pem` and `/etc/kopano/ssl/priv_key.pem`. If found they will used by the secure protocols IMAPS, POP3S and ICALS. Moreover the file ownership will be changed if needed to make them readable by the Kopano services.
+
+#### `SSL_CERTIFICATE_FILE` and `SSL_PRIVATE_KEY_FILE`
+
+If you use other file names or directories, you let the Kopano services know by setting the variables `SSL_CERTIFICATE_FILE=/etc/kopano/ssl/cert.pem` and `SSL_PRIVATE_KEY_FILE=/etc/kopano/ssl/priv_key.pem` on the `docker run` command line or in the `docker-compose.yml` file.
+
+For testing purposes you can create a self-signed certificate using the `openssl` utility, see below. Note that this is not necessary since, when secure protocols are defined, a self-signed certificate and private key will be automatically be created during container startup if they are not found.
+
+```bash
+openssl genrsa -out ssl/priv_key.pem
+openssl req -x509 -utf8 -new -batch -subj "/CN=app" -key ssl/priv_key.pem -out ssl/cert.pem
+```
+
+One way to allow the container to read the certificate and private key is to bind mount the host directory holding the files to the container:
+
+```bash
+docker run -d -name app -v $pwd/ssl:/etc/kopano/ssl mlan/kopano
+```
+
+A other way is to copy them to the container:
+
+```bash
+docker create -name app mlan/kopano
+docker cp ssl/. app:/etc/kopano/ssl
+docker start app
+```
+
+If you copy the files to a running container you need to make sure that the user `kopano` can read them.
+
+### Let’s Encrypt LTS certificates using [Traefik](https://docs.traefik.io/)
+
+[Let’s Encrypt](https://letsencrypt.org/) provide free, automated, authorized certificates when you can demonstrate control over your domain. [Automatic Certificate Management Environment (ACME)](https://en.wikipedia.org/wiki/Automated_Certificate_Management_Environment) is the protocol used for such demonstration. There are many agents and applications that supports ACME, e.g., [certbot](https://certbot.eff.org/). The reverse proxy [Traefik](https://docs.traefik.io/) also supports ACME.
+
+#### `ACME_FILE`, `ACME_POSTHOOK`
+
+The `mlan/kopano` image looks for a file `ACME_FILE=/acme/acme.json` at container startup and every time this file changes certificates within this file are extracted. If the host or domain name of one of those certificates matches `HOSTNAME=$(hostname)` or `DOMAIN=${HOSTNAME#*.}` it will be used by the secure protocols.
+
+Once the certificates and keys have been updated, we run the command in the environment variable `ACME_POSTHOOK="sv restart kopano-gateway kopano-ical"`. Kopano services needs to be restarted to update the LTS parameters. If such automatic reloading is not desired, set `ACME_POSTHOOK=` to empty.
+
+So reusing certificates from Traefik will work out of the box if the `/acme` directory in the Traefik container is also mounted in the `mlan/kopano` container.
+
+```bash
+docker run -d -name proxy -v proxy-acme:/acme traefik
+docker run -d -name app -v proxy-acme:/acme:ro mlan/kopano
+```
+
+Note, if the target certificate Common Name (CN) or Subject Alternate Name (SAN) is changed the container needs to be restarted.
+
+Moreover, do not set any of `SSL_CERTIFICATE_FILE` and `SSL_PRIVATE_KEY_FILE` when using `ACME_FILE`.

 ## Logging `SYSLOG_LEVEL`, `LOG_LEVEL`

@ -317,7 +409,15 @@ Separately, `LOG_LEVEL` controls the logging level of the Kopano services. `LOG_
 | ---- | ---- | ---- | ------- | ------ | ---- | ----- |
 | 0    | 1    | 2    | **3**   | 4      | 5    | 6     |

-## Custom themes
+## Kopano add-ons
+
+### Kopano Archiver
+
+The [Kopano Archiver](https://documentation.kopano.io/kopano_archiver_manual/) provides a Hierarchical Storage Management (HSM) solution for Kopano. With the Kopano Archiver older messages will be automatically moved to slower and thus cheaper storage. The slow storage consists of one or more additional Kopano Archive servers which sole task it is to store archived messages.
+
+Typically the archiver needs its own SQL database. You can configure it using environment variables. When you do, pay attention to [overlapping parameter names](#overlapping-parameter-names). Also the archiver does not run as a daemon but instead you can set up [cron](#cron) jobs. For example, to run the archiver daily with weakly clean-up you can use; `CRONTAB_ENTRY1=0 1 * * * root kopano-archiver -A` and `CRONTAB_ENTRY2=0 3 * * 0 root kopano-archiver -C`.
+
+## WebApp custom themes

 You can easily customize the Kopano WebApp see [New! JSON themes in Kopano WebApp](https://kopano.com/blog/new-json-themes-in-kopano-webapp/). Once you have the files you can install them in your docker container using the receipt below, where we assume that the container name is `mail-app` and that the directory `mytheme` contains the `theme.json` and the other file defining the theme.

@ -329,6 +429,81 @@ docker exec -it mail-app run dc_replace /etc/kopano/webapp/config.php 'define("T

 Please note that it is not possible to rename the directory `/etc/kopano/theme/Custom` within the container without further modifications.

+## WebApp plugins
+
+### S/MIME
+
+[S/MIME](https://en.wikipedia.org/wiki/S/MIME) provides [email encryption](https://en.wikipedia.org/wiki/Email_encryption) guaranteeing the confidentiality and non-repudiation of email. The [S/MIME](https://documentation.kopano.io/webapp_smime_manual/) WebApp plugin is pre-installed.
+
+Using the [demo](#demo) you can easily create a S/MIME certificate you can try out using WebApp.
+
+```sh
+make app-create_smime
+```
+
+### Mobile device management
+
+The [Mobile Device Management](https://documentation.kopano.io/webapp_mdm_manual/) WebApp plugin comes pre-installed. With it you can resync, remove, refresh and even wipe your devices, connected via [Exchange ActiveSync (EAS)](https://en.wikipedia.org/wiki/Exchange_ActiveSync).
+
+## Public folders
+
+There are two type of stores (folders containing communication elements); private and public stores. There can only be one public store. It is the Kopano dagent that places incoming messages into mail boxes, that is the private and public stores.
+
+Public folders are configured by the system admin. Users have them mapped automatically. The shared and public folders can be synced with mobile devices via [Exchange ActiveSync (EAS)](https://wiki.z-hub.io/display/ZP/Sharing+folders+and+Read-only). Each user can manage which shared or public folders to sync with her mobile devices by using the [Mobile Device Management](#mobile-device-management) WebApp plugin.
+
+With the current Kopano implementation, delivering to the public store is configured separately from normal user management. There is the [move to public](https://documentation.kopano.io/kopanocore_administrator_manual/special_kc_configurations.html#move-to-public) plugin which moves incoming messages to a folder in the public store. It has a static configuration and does not support LDAP lookup. 
+
+### Move to public with LDAP lookup
+
+The `mlan/kopano` image include a extended version of the move to public plugin which use LDAP lookup, instead of a static file based lookup. When the plugin [move to public with LDAP](src/kopano/plugin/movetopublicldap.py) is enabled, `DAGENT_PLUGINS=movetopublicldap`, the kopano dagent will do two LDAP queries. The first is to search for an entry/user with matching email address. The second, introduced with this plugin, is to get the public folder from this entry. If found the message will be delivered to the public folder otherwise it will be delivered to the mailbox of the user.
+
+Lets demonstrate how delivery to a public folder is configured. With this LDAP entry:
+
+```yaml
+dn: uid=public,ou=users,dc=example,dc=com
+cn: public
+objectClass: top
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: public
+uid: public
+mail: public@example.com
+kopanoAccount: 1
+kopanoHidden: 1
+kopanoSharedStoreOnly: 1
+kopanoResourceType: publicFolder:Public Stores/public
+```
+
+messages to `public@example.com` will be delivered to the public store in `Public Stores/public`.
+The central [attribute](https://documentation.kopano.io/kopanocore_administrator_manual/appendix_b.html#appendix-b-ldap-attribute-description) is `kopanoResourceType: publicFolder:Public Stores/public`. It contains a token and a folder name. The token match is case sensitive and there must be a colon `:` separating
+the token and the public folder name. The folder name can contain space and
+sub folders, which are distinguished using a forward slash `/`.
+
+The parameters in `/etc/kopano/ldap.cfg` will be used to arrange the LDAP queries.
+The LDAP attribute holding the token and the token itself have the following
+default values, which can be modified in `/etc/kopano/movetopublicldap.cfg` if desired.
+
+```yaml
+ldap_public_folder_attribute = kopanoResourceType
+ldap_public_folder_attribute_token = publicFolder
+```
+
+As with other parameters, environment variables can be used to define them: `LDAP_PUBLIC_FOLDER_ATTRIBUTE=kopanoResourceType` and `LDAP_PUBLIC_FOLDER_ATTRIBUTE_TOKEN=publicFolder`.
+
+## Shared folders
+
+Users can share folders when sufficient permission have been granted. When logged into WebApp with an administrative account (`kopanoAdmin: 1`) you can modify the permissions on users shares and folders. Users can then, when logged into WebApp, open the inbox of other users by selecting `Open Shared Mails`.
+
+The [impersonation](https://wiki.z-hub.io/display/ZP/Impersonation) mechanism allow such shared folders to be synced over [Exchange ActiveSync (EAS)](https://wiki.z-hub.io/display/ZP/Sharing+folders+and+Read-only) too. Each user can manage which shared or public folders to sync with her mobile devices by using the [Mobile Device Management](#mobile-device-management) WebApp plugin.
+
+## Crontab
+
+The `mlan/kopano` has a [cron](https://en.wikipedia.org/wiki/Cron) service activated. You can use environment variables to set up [crontab](https://man7.org/linux/man-pages/man5/crontab.5.html) entries. Any environment variable name staring with `CRONTAB_ENTRY` will be use to add entries to cron.
+
+One trivial example is `CRONTAB_ENTRY_TEST=* * * * * root logger -t cron -p user.notice "SHELL=$$SHELL, PATH=$$PATH"`.
+
+During the initial configuration procedure any `CRONTAB_ENTRY` will add crontab entries to the file `/etc/kopano/docker-crontab`, all the while previously present entries are deleted. This file defines the `PATH` variable so that you don't need to give full path names to commands in the crontab entry. This is, you need to provide the full path names to commands if this `PATH` definition is missing in the `/etc/kopano/docker-crontab` file.
+
 ## Mail transfer agent interaction

 Environment variables can be used to configure where Kopano find the Mail Transfer Agent, such as Postfix. Likewise the Mail Transfer Agent need to know where to forward emails to.
@ -362,13 +537,70 @@ They will run `sa-learn --spam` or `sa-learn --ham`,
 respectively when a message is placed in either `var/lib/kopano/spamd/spam` or
 `var/lib/kopano/spamd/ham`.

+## Migrate old configuration to newer version of Kopano
+
+Sometimes a new version of Kopano breaks compatibility with old configurations. The `mlan/kopano` include some functionality to address such situations. Use`MIGRATE_CONFIG` to try to attempt all or a list of available fixes. `MIGRATE_CONFIG=1 2 3` is an example of a list of fixes and `MIGRATE_CONFIG=all` attempts all fixes.
+
+### `MIGRATE_CONFIG=1` Rejected insecure request as configuration for SECURE_COOKIES is true
+
+Prior to Kopano WebApp version 5.0.0 the parameter was `define("INSECURE_COOKIES", true);` was used to allow HTTP access. Now [`define("SECURE_COOKIES", false);`](https://documentation.kopano.io/webapp_admin_manual/config.html#secure-cookies) is used instead. This fix tries to update the configuration accordingly.
+
+### `MIGRATE_CONFIG=2` Make sure WebApp plugins have configuration files in place
+
+The WebApp plugins S/MIME and MDM has recently been added to the `mlan/kopano` image. Old deployments might not have the related configuration files in place, preventing these plugins from running. This fix places default copies of configuration files in the configuration directory should they be missing.
+
 # Knowledge base

 Here some topics relevant for arranging a mail server are presented.

+## Setup an LDAP directory server
+
+A [Directory Server Agent (DSA)](https://en.wikipedia.org/wiki/Directory_System_Agent) is used to store, organize and present data in a key-value type format. Typically, directories are optimized for lookups, searches, and read operations over write operations, so they function extremely well for data that is referenced often but changes infrequently.
+
+The [Lightweight Directory Access Protocol (LDAP)](https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol), is an open protocol used to store and retrieve data from a hierarchical directory structure. Commonly used to store information about an organization and its assets and users.
+
+[OpenLDAP](https://www.openldap.org/) is a cross platform LDAP based directory service. [Active Directory (AD)](https://en.wikipedia.org/wiki/Active_Directory) is a [directory service](https://en.wikipedia.org/wiki/Directory_service) developed by [Microsoft](https://en.wikipedia.org/wiki/Microsoft) for [Windows domain](https://en.wikipedia.org/wiki/Windows_domain) networks which also uses LDAP.
+
+There are many dockerized OpenLDAP server to choose from. One such example is [mlan/openldap](https://github.com/mlan/docker-openldap).
+
+### LDAP user entry
+
+The data itself in an LDAP system is mainly stored in elements called attributes. Attributes are basically key-value pairs. Unlike in some other systems, the keys have predefined names which are dictated by the objectClasses selected for an entry. An entry is basically a collection of attributes under a name used to describe something; a user for example.
+
+```yaml
+dn: uid=demo,ou=users,dc=example,dc=com
+objectclass: inetOrgPerson
+uid: demo
+```
+
+### Kopano LDAP schema
+
+The Kopano LDAP schema defines additional objectClasses and attributes. These allow the Kopano services, access for example, to controlled on per-user basis. More information on available attributes can be find here, [Fine-tuning user configuration](https://documentation.kopano.io/kopanocore_administrator_manual/configure_kc_components.html?highlight=propmap#fine-tuning-user-configuration).
+
+```shell
+dn: uid=demo,ou=users,dc=example,dc=com
+objectClass: top
+objectClass: inetOrgPerson
+objectClass: kopano-user
+objectClass: posixAccount
+cn: demo
+sn: demo
+uid: demo
+mail: demo@example.com
+uidNumber: 1234
+gidNumber: 1234
+homeDirectory: /home/demo
+telephoneNumber: 0123 123456789
+title: MCP
+kopanoEnabledFeatures: imap
+kopanoEnabledFeatures: pop3
+```
+
+The schema needs to be added to the directory server. The Kopano installation files include the LDAP schema and can be found here `/usr/share/doc/kopano/kopano.ldif.gz`. For more details, see: [Kopano Knowledge Base/Install and optimize OpenLDAP for use with Kopano Groupware Core](https://kb.kopano.io/display/WIKI/Install+and+optimize+OpenLDAP+for+use+with+Kopano+Groupware+Core).
+
 ## Kopano WebApp HTTP access

-The distribution installation of `kopano-webapp` only allow HTTPS access. The `mlan/kopano` image updates the configuration to `define("SECURE_COOKIES", false);` in `/etc/kopano/webapp/config.php` also allowing HTTP access. This can be useful when arranging the `mlan/kopano` container behind a reverse proxy, like [traefik](https://doc.traefik.io/traefik/), which then does the enforcement of HTTPS.
+The distribution installation of `kopano-webapp` only allow HTTPS access. The `mlan/kopano` image updates the configuration to [`define("SECURE_COOKIES", false);`](https://documentation.kopano.io/webapp_admin_manual/config.html#secure-cookies) in `/etc/kopano/webapp/config.php` also allowing HTTP access. This can be useful when arranging the `mlan/kopano` container behind a reverse proxy, like [Traefik](https://doc.traefik.io/traefik/), which then does the enforcement of HTTPS. Also see [`MIGRATE_CONFIG=1` Rejected insecure request as configuration for SECURE_COOKIES is true](#migrate_config1-rejected-insecure-request-as-configuration-for-secure_cookies-is-true).

 ## Mail client configuration

--- a/ROADMAP.md
+++ b/ROADMAP.md
@ -1,58 +1,33 @@
 # Road map

-## kDAV
-
-Consider integrating support for kDAV which provides CalDAV and CardDAV.
-
-## Common configurations
-
-The following directives exist:
-
-```sh
-!include common.cfg
-```
-
-## ACME TLS
-
-Arrange ACME TLS certificates for kopano-gateway (IMAP POP3).
-
 ## Revisit Persistent Data

 Consider consolidating directories which are candidates for persistence under `/srv`.

+- /etc/kopano
+- /var/lib/kopano
+- /var/lib/z-push
+
 ### Kopano Search

 The kopano-search module keeps its database here, /var/lib/kopano/search.
 Consider to also consolidating it under /srv to simplify making it persistent?

+## kDAV
+
+Consider integrating support for kDAV which provides CalDAV and CardDAV.
+
 ## webapp-passwd

 Integrate [webapp-passwd](https://github.com/silentsakky/zarafa-webapp-passwd)?

-## S/MIME
+## kopano-spamd and kopano-search logs

-Install and configure [S/MIME](https://kopano.com/blog/s-mime-plugin-description/)?
+In [KC-1858](https://github.com/Kopano-dev/kopano-core/commit/4a7f833e170167ebfa4f4c55835f8760ce7617f3) we find:

-[S/MIME manual](https://documentation.kopano.io/webapp_smime_manual/).
+> The syslog log method does not work correctly and thus this change
+> disables it. Until it is fixed, Python services do not support
+> the syslog log_method. Additionally an environment variable is
+> added, which allow to lift this restriction for testing when it
+> it set.

-## MDM
-
-Install and configure [MDM](https://documentation.kopano.io/webapp_mdm_manual/)?
-With the MDM plugin you can resync, remove, refresh and even wipe your device.
-
-## Improve Health Check?
-
-Verify the user anonymously.
-```bash
-ldapsearch -h dockerhost -xLLL -b dc=example,dc=com '(kopanoAccount=1)'
-```
-
-Check if kopano can get the user from LDAP
-```bash
-kopano-admin -l
-```
-check that apache and mysql is running
-```bash
-apache2ctl status
-mysqlcheck -A
-```
--- a/demo/.env
+++ b/demo/.env
@ -1,22 +1,34 @@
 COMPOSE_PROJECT_NAME=demo
-SYSLOG_LEVEL=5
+SYSLOG_LEVEL=7
+LOG_LEVEL=6
+AD_DEBUG=stats
+AD_ADM_CN=admin
+AD_ADM_PW=admin
+AD_ADM_TEL=555-540-9637
+AD_ADM_TIT=System Admin
+AD_BASE=dc=example,dc=com
+AD_GRP_CN=team
+AD_GRP_OB=kopano-group
+AD_GRP_OU=groups
+AD_PUB_CN=public
+AD_ROOT_CN=admin
+AD_ROOT_PW=secret
+AD_SHR_CN=shared
+AD_USR_AS=trial
+AD_USR_CN=demo
+AD_USR_OB=kopano-user
+AD_USR_OU=users
+AD_USR_PW=demo
+AD_USR_TEL=555-439-2736
+AD_USR_TIT=First User
+DKIM_SELECTOR=default
 MAIL_DOMAIN=example.com
 MAIL_SRV=mx
-REGEX_ALIAS='/([^+]+)[+-].*@(.+)/ $1@$2'
-DKIM_SELECTOR=default
-SA_TAG_LEVEL_DEFLT=-999
-SA_DEBUG=0
-LOG_LEVEL=5
-RAZOR_REGISTRATION=
-LDAP_BASE=dc=example,dc=com
-LDAP_USEROU=users
-LDAP_USEROBJ=posixAccount
-LDAP_USERFLT=
-LDAP_GROUPOU=groups
-LDAP_GROUPOBJ=posixGroup
-LDAP_TEST_USER=demo
-LDAP_TEST_PASSWD=demo
-MYSQL_ROOT_PASSWORD=secret
 MYSQL_DATABASE=kopano
-MYSQL_USER=kopano
 MYSQL_PASSWORD=secret
+MYSQL_ROOT_PASSWORD=secret
+MYSQL_USER=kopano
+RAZOR_REGISTRATION=
+REGEX_ALIAS='/([^+]+)[+-].*@(.+)/ $1@$2'
+SA_DEBUG=0
+SA_TAG_LEVEL_DEFLT=-999
--- a/demo/.gitignore
+++ b/demo/.gitignore
@ -0,0 +1,2 @@
+ssl
+utils-container.mk
--- a/demo/Makefile
+++ b/demo/Makefile
@ -1,21 +1,46 @@
+# Makefile
+#
+# demo
+#
+
 -include    *.mk .env .init.env

-srv_list ?= auth app db mta
-curl_dbg ?= -v
-_ip       = $(shell docker inspect -f \
-	'{{range .NetworkSettings.Networks}}{{println .IPAddress}}{{end}}' \
-	$(1) | head -n1)
+SRV_LIST ?= auth app db mta

-.PHONY:
+AD_DOM   ?= $(call ad_sub_dot, $(AD_BASE))
+AD_DC    ?= $(call ad_cut_dot, 1, 1, $(AD_DOM))
+
+SSL_O     = $(MAIL_DOMAIN)
+SSL_MAIL  = auto
+SSL_PASS  = $(AD_USR_PW)
+#SSL_TRST  = $(SSL_SMIME)
+
+NET_NAME ?= $(COMPOSE_PROJECT_NAME)_backend
+CURL_OPT ?= -s -v
+TSSL_CMD ?= docker run -i --rm --network $(NET_NAME) drwetter/testssl.sh
+CURL_CMD ?= curl
+webb_cmd ?= firefox $(1) &
+APP_NAME  = app
+AUT_NAME  = auth
+AUW_NAME  = auth-web
+DB_NAME   = db
+DBW_NAME  = db-web
+MTA_NAME  = mta
+APP_FQDN ?= $(call dkr_srv_ip,$(APP_NAME))
+AUT_FQDN ?= $(call dkr_srv_ip,$(AUT_NAME))
+AUW_FQDN ?= $(call dkr_cnt_ip,$(AUW_NAME))
+DB_FQDN  ?= $(call dkr_srv_ip,$(DB_NAME))
+DBW_FQDN ?= $(call dkr_cnt_ip,$(DBW_NAME))
+MTA_FQDN ?= $(call dkr_srv_ip,$(MTA_NAME))
+
+MAIL_FROM ?= test@my-domain.biz

 variables:
 	make -pn | grep -A1 "^# makefile"| grep -v "^#\|^--" | sort | uniq

-test: mta-test_quiet mta-test_smtp
+test: all-test_quiet mta-test_smtp

-init: auth-init db-up mta-up app-up
-
-auth-init: auth-up wait_11 auth-mod_hash auth-mod_index auth-add_user
+init: up auth-init db-init app-restart mta-init app-init

 ps:
 	docker-compose ps
@ -26,7 +51,7 @@ up:
 down:
 	docker-compose down

-destroy:
+destroy: auth-web-down db-web-down all-destroy_smime
 	docker-compose down -v

 config:
@ -38,50 +63,192 @@ logs:
 images:
 	docker-compose images

-$(addsuffix -up,$(srv_list)):
+$(addsuffix -up,$(SRV_LIST)):
 	docker-compose up -d $(patsubst %-up,%,$@)

-$(addsuffix -down,$(srv_list)):
-	docker-compose rm -s $(patsubst %-down,%,$@)
+$(addsuffix -down,$(SRV_LIST)):
+	docker-compose rm -sf $(patsubst %-down,%,$@)

-$(addsuffix -restart,$(srv_list)):
+$(addsuffix -restart,$(SRV_LIST)):
 	docker-compose restart $(patsubst %-restart,%,$@)

-$(addsuffix -renew,$(srv_list)):
+$(addsuffix -renew,$(SRV_LIST)):
 	docker-compose rm -s $(patsubst %-renew,%,$@)
 	docker-compose up -d $(patsubst %-renew,%,$@)

-$(addsuffix -top,$(srv_list)):
+$(addsuffix -top,$(SRV_LIST)):
 	docker-compose top $(patsubst %-top,%,$@)

-$(addsuffix -logs,$(srv_list)):
+$(addsuffix -logs,$(SRV_LIST)):
 	docker-compose logs $(patsubst %-logs,%,$@)

-$(addsuffix -sh,$(srv_list)):
-	docker-compose exec $(patsubst %-sh,%,$@) sh
+$(addsuffix -pull,$(SRV_LIST)):
+	docker-compose pull $(patsubst %-pull,%,$@)

-$(addsuffix -env,$(srv_list)):
+$(addsuffix -sh,$(SRV_LIST)):
+	docker-compose exec $(patsubst %-sh,%,$@) sh -c 'exec $$(getent passwd root | sed "s/.*://g")'
+
+$(addsuffix -env,$(SRV_LIST)):
 	docker-compose exec $(patsubst %-env,%,$@) env

-$(addsuffix -sv,$(srv_list)):
+$(addsuffix -sv,$(SRV_LIST)):
 	docker-compose exec $(patsubst %-sv,%,$@) sh -c 'sv status $$SVDIR/*'

-$(addsuffix -diff,$(srv_list)):
+$(addsuffix -diff,$(SRV_LIST)):
 	docker container diff $(COMPOSE_PROJECT_NAME)_$(patsubst %-diff,%,$@)_1

 wait_%:
 	sleep $*

-web:
-	firefox localhost:8008 &
+web: app-web
+
+auth-init: wait_3 auth-mod_conf auth-add_schema auth-add_data
+
+export define LDIF_MOD_CONF
+dn: olcDatabase={-1}frontend,cn=config
+changetype: modify
+add: olcPasswordHash
+olcPasswordHash: {CRYPT}
+
+dn: cn=config
+changetype: modify
+add: olcPasswordCryptSaltFormat
+olcPasswordCryptSaltFormat: $$6$$%.16s
+
+dn: olcDatabase={1}mdb,cn=config
+changetype: modify
+add: olcDbIndex
+olcDbIndex: cn,ou,uid,mail eq
+endef
+
+export define LDIF_ADD_DATA
+dn: $(AD_BASE)
+objectClass: organization
+objectClass: dcObject
+dc: $(AD_DC)
+o: $(AD_DOM)
+
+dn: ou=$(AD_USR_OU),$(AD_BASE)
+ou: $(AD_USR_OU)
+objectClass: organizationalUnit
+
+dn: ou=$(AD_GRP_OU),$(AD_BASE)
+ou: $(AD_GRP_OU)
+objectClass: organizationalUnit
+
+dn: cn=$(AD_GRP_CN),ou=$(AD_GRP_OU),$(AD_BASE)
+cn: $(AD_GRP_CN)
+objectClass: groupOfNames
+objectClass: kopano-group
+member: uid=$(AD_ADM_CN),ou=$(AD_USR_OU),$(AD_BASE)
+member: uid=$(AD_USR_CN),ou=$(AD_USR_OU),$(AD_BASE)
+mail: $(AD_GRP_CN)@$(MAIL_DOMAIN)
+
+dn: uid=$(AD_ADM_CN),ou=$(AD_USR_OU),$(AD_BASE)
+changetype: add
+cn: $(AD_ADM_CN)
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: $(AD_ADM_CN)
+uid: $(AD_ADM_CN)
+mail: $(AD_ADM_CN)@$(MAIL_DOMAIN)
+userPassword: $(AD_ADM_PW)
+telephoneNumber: $(AD_ADM_TEL)
+title: $(AD_ADM_TIT)
+kopanoAccount: 1
+kopanoAdmin: 1
+kopanoEnabledFeatures: imap
+kopanoEnabledFeatures: pop3
+
+dn: uid=$(AD_USR_CN),ou=$(AD_USR_OU),$(AD_BASE)
+changetype: add
+cn: $(AD_USR_CN)
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: $(AD_USR_CN)
+uid: $(AD_USR_CN)
+mail: $(AD_USR_CN)@$(MAIL_DOMAIN)
+userPassword: $(AD_USR_PW)
+telephoneNumber: $(AD_USR_TEL)
+title: $(AD_USR_TIT)
+kopanoAccount: 1
+kopanoAliases: $(AD_USR_AS)@$(MAIL_DOMAIN)
+kopanoEnabledFeatures: imap
+kopanoEnabledFeatures: pop3
+
+dn: uid=$(AD_SHR_CN),ou=$(AD_USR_OU),$(AD_BASE)
+cn: $(AD_SHR_CN)
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: $(AD_SHR_CN)
+uid: $(AD_SHR_CN)
+mail: $(AD_SHR_CN)@$(MAIL_DOMAIN)
+kopanoAccount: 1
+kopanoSharedStoreOnly: 1
+
+dn: uid=$(AD_PUB_CN),ou=$(AD_USR_OU),$(AD_BASE)
+cn: $(AD_PUB_CN)
+objectClass: inetOrgPerson
+objectClass: kopano-user
+sn: $(AD_PUB_CN)
+uid: $(AD_PUB_CN)
+mail: $(AD_PUB_CN)@$(MAIL_DOMAIN)
+kopanoAccount: 1
+kopanoHidden: 1
+kopanoSharedStoreOnly: 1
+kopanoResourceType: publicFolder:Public Stores/public
+endef
+
+define smtp_mail
+@printf "From: <$(2)>\nTo: <$(3)>\nDate: $$(date -R)\nSubject: $(4)\
+\n\nGreat news! You can receive email.\n" | tee /dev/tty \
+| $(CURL_CMD) $(1) -T - --mail-from $(2) --mail-rcpt $(3) $(CURL_OPT)
+endef
+
+define lmtp_mail
+printf "LHLO mx\nMAIL FROM: <$(2)>\nRCPT TO: <$(3)>\nDATA\
+\nFrom: <$(2)>\nTo: <$(3)>\nDate: $$(date -R)\nSubject: $(4)\
+\n\nGreat news! You can receive email.\n.\nQUIT\n" | tee /dev/tty \
+| $(CURL_CMD) $(1) -T - $(CURL_OPT)
+endef
+
+export define MAKE_UTILS_CONTAINER
+CURL_CMD ?= docker run -i --rm --network $(NET_NAME) curlimages/curl
+webb_cmd ?= docker run -d --rm --network $(NET_NAME) \
+-e DISPLAY=$$$$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix \
+-v /etc/localtime:/etc/localtime:ro -v $$$$(pwd)/ssl:/ssl \
+kennethkl/firefox $$(1)
+APP_FQDN ?= $(APP_NAME)
+AUT_FQDN ?= $(AUT_NAME)
+AUW_FQDN ?= $(AUW_NAME)
+DB_FQDN  ?= $(DB_NAME)
+DBW_FQDN ?= $(DBW_NAME)
+MTA_FQDN ?= $(MTA_NAME)
+endef
+
+utils-container:
+	echo "$$MAKE_UTILS_CONTAINER" > utils-container.mk
+
+utils-default:
+	rm -f utils-container.mk
+
+auth-mod_conf:
+	echo "$$LDIF_MOD_CONF" | docker-compose exec -T auth ldapmodify -Q
+
+auth-add_data:
+	echo "$$LDIF_ADD_DATA" | docker-compose exec -T auth ldapadd -Q
+
+auth-add_schema:
+	docker-compose exec app zcat /usr/share/doc/kopano/kopano.ldif.gz \
+	| docker-compose exec -T auth ldapadd -Q

 auth-show_conf:
-	docker-compose exec auth ldap search -b cn=config "(cn=config)"
-	docker-compose exec auth ldap search -b cn=config olcDatabase={-1}frontend
-	docker-compose exec auth ldap search -b cn=config olcDatabase={1}mdb
+	docker-compose exec auth ldapsearch -QLLLb cn=config "(cn=config)"
+	docker-compose exec auth ldapsearch -QLLLb cn=config olcDatabase={-1}frontend
+	docker-compose exec auth ldapsearch -QLLLb cn=config olcDatabase={1}mdb

-auth-show_user:
-	docker-compose exec auth ldap search -b "$(LDAP_BASE)"
+auth-show_data:
+	docker-compose exec auth ldapsearch -QLLL

 auth-show_cat0:
 	docker-compose exec auth slapcat -n0
@ -89,39 +256,44 @@ auth-show_cat0:
 auth-show_cat1:
 	docker-compose exec auth slapcat -n1

-auth-add_user:
-	printf "dn: ou=$(LDAP_USEROU),$(LDAP_BASE)\nchangetype: add\nobjectClass: organizationalUnit\nobjectClass: top\nou: $(LDAP_USEROU)\n\ndn: ou=$(LDAP_GROUPOU),$(LDAP_BASE)\nchangetype: add\nobjectClass: organizationalUnit\nobjectClass: top\nou: $(LDAP_GROUPOU)\n\ndn: uid=$(LDAP_TEST_USER),ou=$(LDAP_USEROU),$(LDAP_BASE)\nchangetype: add\nobjectClass: top\nobjectClass: inetOrgPerson\nobjectClass: $(LDAP_USEROBJ)\ncn: $(LDAP_TEST_USER)\nsn: $(LDAP_TEST_USER)\nuid: $(LDAP_TEST_USER)\nmail: $(LDAP_TEST_USER)@$(MAIL_DOMAIN)\nuidNumber: 1234\ngidNumber: 1234\nhomeDirectory: /home/$(LDAP_TEST_USER)\nuserPassword: $(LDAP_TEST_PASSWD)\n" \
-	| docker-compose exec -T auth ldap modify
+auth-web: auth-web-up
+	sleep 2
+	$(call webb_cmd,http://$(AUW_FQDN))

-auth-mod_index:
-	printf "dn: olcDatabase={1}mdb,cn=config\nchangetype: modify\nadd: olcDbIndex\nolcDbIndex: cn,ou,uid,mail eq\n" \
-	| docker-compose exec -T auth ldap modify
+auth-web-up:
+	docker run -d --name $(AUW_NAME) --network $(NET_NAME) \
+	-e PHPLDAPADMIN_LDAP_HOSTS=auth -e PHPLDAPADMIN_HTTPS=false \
+	osixia/phpldapadmin || true

-auth-mod_hash:
-	printf "dn: olcDatabase={-1}frontend,cn=config\nchangetype: modify\nadd: olcPasswordHash\nolcPasswordHash: {CRYPT}\n\ndn: cn=config\nchangetype: modify\nadd: olcPasswordCryptSaltFormat\nolcPasswordCryptSaltFormat: \$$6\$$%%.16s\n" \
-	| docker-compose exec -T auth ldap modify
+auth-web-down:
+	docker rm -f $(AUW_NAME) || true
+
+mta-init:

 mta-bayes:
 	docker-compose exec mta sh -c 'rm -f bayesian.database.gz && wget http://artinvoice.hu/spams/bayesian.database.gz && gunzip bayesian.database.gz && sa-learn --restore bayesian.database && chown -R amavis: /var/amavis/.spamassassin && rm -rf bayesian.database'

-mta-test_quiet:
-	$(eval curl_dbg := )
-
 mta-test_smtp:
-	printf "From: A tester <test@example.biz>\nTo: <$(LDAP_TEST_USER)@$(MAIL_DOMAIN)>\nDate: $$(date)\nSubject: A SMTP test message\n\nGreat news! You can receive email.\n" \
-	| curl smtp://localhost -s -T - $(curl_dbg) \
-	--mail-from test@example.biz --mail-rcpt $(LDAP_TEST_USER)@$(MAIL_DOMAIN)
+	$(call smtp_mail,smtp://$(MTA_FQDN),$(MAIL_FROM),$(AD_USR_CN)@$(MAIL_DOMAIN),A SMTP test message.)

-mta-test_smtp2:
-	printf "From: A info tester <test-info@example.biz>\nTo: <$(LDAP_TEST_USER)-info@$(MAIL_DOMAIN)>\nDate: $$(date)\nSubject: A SMTP test message \n\nGreat news! $(LDAP_TEST_USER)-info@$(MAIL_DOMAIN) can also receive email.\n" \
-	| curl smtp://localhost -s -T - $(curl_dbg) \
-	--mail-from test@example.biz --mail-rcpt $(LDAP_TEST_USER)@$(MAIL_DOMAIN)
+mta-test_regexp:
+	$(call smtp_mail,smtp://$(MTA_FQDN),$(MAIL_FROM),$(AD_USR_CN)+info@$(MAIL_DOMAIN),A regexp SMTP test message.)
+
+mta-test_smtps:
+	$(call smtp_mail,smtps://$(MTA_FQDN),$(MAIL_FROM),$(AD_USR_CN)@$(MAIL_DOMAIN),A secure SMTPS test message.) \
+	-k -u $(AD_USR_CN):$(AD_USR_PW)
+
+mta-test_shared: all-test_quiet
+	$(call smtp_mail,smtp://$(MTA_FQDN),$(MAIL_FROM),$(AD_SHR_CN)@$(MAIL_DOMAIN),A shared SMTP test message.)
+
+mta-test_public: all-test_quiet
+	$(call smtp_mail,smtp://$(MTA_FQDN),$(MAIL_FROM),$(AD_PUB_CN)@$(MAIL_DOMAIN),A public SMTP test message.)

 mta-razor:
 	docker-compose exec mta run amavis_register_razor

 mta-apk_list:
-	docker-compose exec mta /bin/sh -c 'for pkg in $$(apk info 2>/dev/null); do printf "%9s  %s\n" $$(apk info -s $$pkg 2>/dev/null | sed -n "2{p;q}") $$pkg; done | sort'
+	docker-compose exec mta /bin/sh -c 'for pkg in $$(apk info 2>/dev/null); do printf "%9s %3s %s\n" $$(apk info -s $$pkg 2>/dev/null | sed -n "2{p;q}") $$pkg; done | sort | sort -k 2,2'

 mta-quarantine_list:
 	docker-compose exec mta amavis-ls
@ -132,11 +304,11 @@ mta-freshclam_nodns:
 mta-clamdtop:
 	docker-compose exec mta clamdtop

-mta-debugtools:
+mta-tools:
 	docker-compose exec mta apk --no-cache --update add \
-	nano less lsof htop openldap-clients bind-tools iputils strace
+	nano less lsof htop openldap-clients bind-tools iputils strace iproute2

-mta-htop: mta-debugtools
+mta-htop: mta-tools
 	docker-compose exec mta htop

 mta-encrypt:
@ -155,66 +327,112 @@ mta-show_mailq:
 mta-flush_mailq:
 	docker-compose exec mta postqueue -f

-mta-hostaddr:
-	$(eval myhost := $(call _ip,$(COMPOSE_PROJECT_NAME)_mta_1))
-
 mta-test_auth:
-	docker-compose exec mta doveadm auth test $(LDAP_TEST_USER) $(LDAP_TEST_PASSWD)
+	docker-compose exec mta doveadm auth test $(AD_USR_CN) $(AD_USR_PW)

-mta-test_imap: mta-hostaddr
-	curl imap://$(myhost) -X CAPABILITY
-	curl imap://$(myhost) -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD)
+mta-test_imaps:
+	$(CURL_CMD) imaps://$(MTA_FQDN)//inbox -X "fetch 1 all" \
+	--ssl --anyauth -k -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)

-mta-test_rimap:
-	docker-compose exec mta curl imap://app -X CAPABILITY
-	docker-compose exec mta curl imap://app -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD)
+mta-man:
+	docker-compose exec mta apk --no-cache --update add man-db man-pages \
+	postfix-doc cyrus-sasl-doc dovecot-doc spamassassin-doc clamav-doc razor-doc

-mta-test_ldap: mta-debugtools
-	docker-compose exec mta ldapsearch -H ldap://auth:389 -xLLL -s base namingContexts
+db-init:

 db-test:
 	docker-compose exec db mysqlshow -u $(MYSQL_USER) $(MYSQL_DATABASE) -p$(MYSQL_PASSWORD)

-app-debugtools:
+db-web: db-web-up
+	sleep 2
+	$(call webb_cmd,http://$(DBW_FQDN))
+
+db-web-up:
+	docker run -d --name $(DBW_NAME) --network $(NET_NAME) \
+	-e PMA_HOST=db phpmyadmin/phpmyadmin || true
+
+db-web-down:
+	docker rm -f $(DBW_NAME) || true
+
+app-init: app-public_store app-create_smime
+
+app-tools:
 	docker-compose exec app apt-get update
 	docker-compose exec app apt-get install --yes \
-	less nano ldap-utils htop net-tools lsof iputils-ping strace
+	less nano ldap-utils htop net-tools lsof iputils-ping dnsutils strace

-app-htop: app-debugtools
+app-htop: app-tools
 	docker-compose exec app htop

-app-man_server:
-	docker-compose exec app man kopano-server.cfg
-
-app-man_ldap:
-	docker-compose exec app man kopano-ldap.cfg
-
-app-hostaddr:
-	$(eval myhost := $(call _ip,$(COMPOSE_PROJECT_NAME)_app_1))
-
 app-test_smtp: mta-test_smtp

-app-test_lmtp: app-hostaddr
-	printf "LHLO mx\nMAIL FROM: <test@example.biz>\nRCPT TO: <$(LDAP_TEST_USER)@$(MAIL_DOMAIN)>\nDATA\nFrom: A tester <test@example.biz>\nTo: <$(LDAP_TEST_USER)@$(MAIL_DOMAIN)>\nDate: $$(date)\nSubject: A LMTP test message from me to you\n\nDelete me, please \n.\nQUIT\n" | nc -C $(myhost) 2003
+app-test_lmtp:
+	$(call lmtp_mail,telnet://$(APP_FQDN):2003,$(MAIL_FROM),$(AD_USR_CN)@$(MAIL_DOMAIN),A LMTP test message.)

-app-test_imap: app-hostaddr
-	curl imap://$(myhost) -X CAPABILITY
-	curl imap://$(myhost) -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD)
+app-test_all: all-test_muted $(addprefix app-test_,imap pop3 ical imaps pop3s icals)

-app-test_pop3: app-hostaddr
-	curl pop3://$(myhost) -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD)
+app-test_imap:
+	$(CURL_CMD) imap://$(APP_FQDN) -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)

-app-test_ical: app-hostaddr
-	curl http://$(myhost):8080 -u $(LDAP_TEST_USER):$(LDAP_TEST_PASSWD)
+app-test_imaps:
+	$(CURL_CMD) imaps://$(APP_FQDN) -k -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
+
+app-test_pop3:
+	$(CURL_CMD) pop3://$(APP_FQDN) -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
+
+app-test_pop3s:
+	$(CURL_CMD) pop3s://$(APP_FQDN) -k -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
+
+app-test_ical:
+	$(CURL_CMD) http://$(APP_FQDN):8080 -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
+
+app-test_icals:
+	$(CURL_CMD) https://$(APP_FQDN):8443 -k -u $(AD_USR_CN):$(AD_USR_PW) $(CURL_OPT)
+
+app-test_tls:
+	$(TSSL_CMD) $(APP_FQDN):993 || true
+
+app-web:
+	$(call webb_cmd,http://$(APP_FQDN))
+
+app-test_oof1:
+	docker-compose exec app kopano-oof -u $(AD_USR_CN) -m 1 -t "Dunno when I return"
+
+app-test_oof0:
+	docker-compose exec app kopano-oof -u $(AD_USR_CN) -m 0

 app-show_user1:
-	docker-compose exec app kopano-admin -l
+	docker-compose exec app kopano-admin --details $(AD_USR_CN)

-app-show_user2: app-debugtools
-	docker-compose exec app ldapsearch -H ldap://auth:389 -xLLL -b $(LDAP_BASE) '*'
+app-show_user2: app-tools
+	docker-compose exec app ldapsearch -H ldap://auth:389 -xLLL -b $(AD_BASE) '*'

 app-show_sync:
 	docker-compose exec app z-push-top

 app-create_store:
-	docker-compose exec app kopano-admin --create-store $(LDAP_TEST_USER)
+	docker-compose exec app kopano-admin --create-store $(AD_USR_CN)
+
+app-public_store:
+	docker-compose exec app kopano-storeadm -P
+
+#app-add_user:
+#	docker-compose exec app kopano-admin -c $(AD_USR_CN) -p $(AD_USR_PW) \
+#	-e $(AD_USR_CN)@$(MAIL_DOMAIN) -f $(AD_USR_CN) -a 1
+
+$(addprefix app-parms_,archiver dagent gateway ical ldap search server spamd spooler):
+	docker-compose exec app run list_parms $(patsubst app-parms_%,%,$@)
+
+app-create_smime: all-create_smime
+	docker cp ssl/ca.crt $(call dkr_srv_cnt,app):/usr/local/share/ca-certificates/$(MAIL_DOMAIN)_CA.crt
+	docker-compose exec app update-ca-certificates
+
+all-test_quiet:
+	$(eval CURL_OPT := -s -S )
+
+all-test_muted:
+	$(eval CURL_OPT := -s -S >/dev/null || true)
+
+all-create_smime: ssl/$(AD_USR_CN).p12
+
+all-destroy_smime: ssl-destroy
--- a/demo/ad.mk
+++ b/demo/ad.mk
@ -0,0 +1 @@
+../test/ad.mk
--- a/demo/dkr.mk
+++ b/demo/dkr.mk
@ -0,0 +1 @@
+../test/dkr.mk
--- a/demo/docker-compose.yml
+++ b/demo/docker-compose.yml
@ -5,31 +5,41 @@ services:
    image: mlan/kopano
    networks:
      - backend
-    ports:
-      - "127.0.0.1:8008:80"    # WebApp & EAS (alt. HTTP)
-      - "127.0.0.1:110:110"    # POP3 (not needed if all devices can use EAS)
-      - "127.0.0.1:143:143"    # IMAP (not needed if all devices can use EAS)
-      - "127.0.0.1:8080:8080"  # CalDAV (not needed if all devices can use EAS)
+#    ports: # Uncomment to expose ports to host interfaces
+#      - "80:80"    # WebApp & EAS (alt. HTTP)
+#      - "143:143"    # IMAP (not needed if all devices can use EAS)
+#      - "110:110"    # POP3 (not needed if all devices can use EAS)
+#      - "8080:8080"  # ICAL (not needed if all devices can use EAS)
+#      - "993:993"    # IMAPS (not needed if all devices can use EAS)
+#      - "995:995"    # POP3S (not needed if all devices can use EAS)
+#      - "8443:8443"  # ICALS (not needed if all devices can use EAS)
    depends_on:
      - auth
      - db
      - mta
    environment: # Virgin config, ignored on restarts unless FORCE_CONFIG given.
      - USER_PLUGIN=ldap
-      - LDAP_URI=ldap://auth:389/
+      - LDAP_URI=ldap://auth/
      - MYSQL_HOST=db
      - SMTP_SERVER=mta
-      - LDAP_SEARCH_BASE=${LDAP_BASE-dc=example,dc=com}
-      - LDAP_USER_TYPE_ATTRIBUTE_VALUE=${LDAP_USEROBJ-posixAccount}
-      - LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=${LDAP_GROUPOBJ-posixGroup}
+      - LDAP_SEARCH_BASE=${AD_BASE-dc=example,dc=com}
+      - LDAP_USER_TYPE_ATTRIBUTE_VALUE=${AD_USR_OB-kopano-user}
+      - LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=${AD_GRP_OB-kopano-group}
+      - LDAP_GROUPMEMBERS_ATTRIBUTE_TYPE=dn
+      - LDAP_PROPMAP=
+      - DAGENT_PLUGINS=movetopublicldap
      - MYSQL_DATABASE=${MYSQL_DATABASE-kopano}
      - MYSQL_USER=${MYSQL_USER-kopano}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD-secret}
-      - POP3_LISTEN=*:110                       # also listen to eth0
      - IMAP_LISTEN=*:143                       # also listen to eth0
+      - POP3_LISTEN=*:110                       # also listen to eth0
      - ICAL_LISTEN=*:8080                      # also listen to eth0
-      - DISABLED_FEATURES=${DISABLED_FEATURES-} # also enable IMAP and POP3
+      - IMAPS_LISTEN=*:993                      # enable TLS
+      - POP3S_LISTEN=*:995                      # enable TLS
+      - ICALS_LISTEN=*:8443                     # enable TLS
+      - PLUGIN_SMIME_USER_DEFAULT_ENABLE_SMIME=true
      - SYSLOG_LEVEL=${SYSLOG_LEVEL-3}
+      - LOG_LEVEL=${LOG_LEVEL-3}
    volumes:
      - app-conf:/etc/kopano
      - app-atch:/var/lib/kopano/attachments
@ -44,8 +54,9 @@ services:
    hostname: ${MAIL_SRV-mx}.${MAIL_DOMAIN-example.com}
    networks:
      - backend
-    ports:
-      - "127.0.0.1:25:25"      # SMTP
+#    ports: # # Uncomment to expose ports to host interfaces
+#      - "25:25"      # SMTP
+#      - "465:465"    # SMTPS authentication required
    depends_on:
      - auth
    environment: # Virgin config, ignored on restarts unless FORCE_CONFIG given.
@ -55,8 +66,9 @@ services:
      - SMTP_RELAY_HOSTAUTH=${SMTP_RELAY_HOSTAUTH-}
      - SMTP_TLS_SECURITY_LEVEL=${SMTP_TLS_SECURITY_LEVEL-}
      - SMTP_TLS_WRAPPERMODE=${SMTP_TLS_WRAPPERMODE-no}
-      - LDAP_USER_BASE=ou=${LDAP_USEROU-users},${LDAP_BASE-dc=example,dc=com}
-      - LDAP_QUERY_FILTER_USER=(&(objectclass=${LDAP_USEROBJ-posixAccount})(mail=%s))
+      - SMTPD_USE_TLS=yes
+      - LDAP_USER_BASE=ou=${AD_USR_OU-users},${AD_BASE-dc=example,dc=com}
+      - LDAP_QUERY_FILTER_USER=(&(objectclass=${AD_USR_OB-kopano-user})(mail=%s))
      - LDAP_QUERY_ATTRS_PASS=uid=user
      - REGEX_ALIAS=${REGEX_ALIAS-}
      - DKIM_SELECTOR=${DKIM_SELECTOR-default}
@ -91,8 +103,10 @@ services:
    image: mlan/openldap
    networks:
      - backend
+    command: --root-cn ${AD_ROOT_CN-admin} --root-pw ${AD_ROOT_PW-secret}
    environment:
-      - LDAP_LOGLEVEL=parse
+      - LDAPBASE=${AD_BASE-dc=example,dc=com}
+      - LDAPDEBUG=${AD_DEBUG-parse}
    volumes:
      - auth:/srv
      - /etc/localtime:/etc/localtime:ro        # Use host timezone
--- a/demo/ssl.mk
+++ b/demo/ssl.mk
@ -0,0 +1 @@
+../test/ssl.mk
--- a/src/acme/bin/acme-extract.sh
+++ b/src/acme/bin/acme-extract.sh
@ -0,0 +1,193 @@
+#!/usr/bin/env sh
+# Copyright (c) 2017 Brian 'redbeard' Harrington <redbeard@dead-city.org>
+#
+# acme-export.sh - A simple utility to explode a Traefik acme.json file into a
+#                directory of certificates and a private key
+#
+# Usage - acme-export.sh /etc/traefik/acme.json /etc/ssl/
+#
+# Dependencies -
+#   util-linux
+#   openssl
+#   jq
+# The MIT License (MIT)
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+
+# Exit codes:
+# 1 - A component is missing or could not be read
+# 2 - There was a problem reading acme.json
+# 4 - The destination certificate directory does not exist
+# 8 - Missing private key
+
+#set -o errexit
+##set -o pipefail
+#set -o nounset
+#set -o verbose
+
+
+#
+# configuration
+#
+. docker-common.sh
+
+DOCKER_ACME_SSL_DIR=${DOCKER_ACME_SSL_DIR-/etc/ssl/acme}
+ACME_FILE=${ACME_FILE-/acme/acme.json}
+
+CMD_DECODE_BASE64="base64 -d"
+
+#
+# functions
+#
+usage() { echo "$(basename $0) <path to acme> <destination cert directory>" ;}
+
+test_args() {
+	# when called by inotifyd the first argument is the single character
+	# event descriptor, lets drop it
+	dc_log 7 "Called with args $@"
+	[ $# -ge 0 ] && [ ${#1} -eq 1 ] && shift
+	readonly acmefile="${1-$ACME_FILE}"
+	readonly certdir="${2-$DOCKER_ACME_SSL_DIR}"
+}
+
+test_dependencies() {
+	# Allow us to exit on a missing jq binary
+	if dc_is_installed jq; then
+		dc_log 7 "The package jq is installed."
+	else
+		dc_log 4 "You must have the binary jq to use this."
+		exit 1
+	fi
+}
+
+test_acmefile() {
+	if [ ! -r "${acmefile}" ]; then
+		dc_log 4 "There was a problem reading from (${acmefile}). We need to read this file to explode the JSON bundle... exiting."
+		exit 2
+	fi
+}
+
+test_certdir() {
+	if [ ! -d "${certdir}" ]; then
+		dc_log 4 "Path ${certdir} does not seem to be a directory. We need a directory in which to explode the JSON bundle... exiting."
+		exit 4
+	fi
+}
+
+make_certdirs() {
+	# If they do not exist, create the needed subdirectories for our assets
+	# and place each in a variable for later use, normalizing the path
+	mkdir -p "${certdir}/certs" "${certdir}/private"
+	pdir="${certdir}/private"
+	cdir="${certdir}/certs"
+}
+
+bad_acme() {
+	dc_log 4 "There was a problem parsing your acme.json file."
+	exit 2
+}
+
+read_letsencryptkey() {
+	# look for key assuming acme v2 format
+	priv=$(jq -e -r '.[].Account.PrivateKey' "${acmefile}" 2>/dev/null)
+	if [ $? -eq 0 ]; then
+		acmeversion=2
+		dc_log 7 "Using acme v2 format, the PrivateKey was found in ${acmefile}"
+	else
+		# look for key assuming acme v1 format
+		priv=$(jq -e -r '.Account.PrivateKey' "${acmefile}" 2>/dev/null)
+		if [ $? -eq 0 ]; then
+			acmeversion=1
+			dc_log 7 "Using acme v1 format, the PrivateKey was found in ${acmefile}"
+		else
+			dc_log 4 "There didn't seem to be a private key in ${acmefile}. Please ensure that there is a key in this file and try again."
+			exit 2
+		fi
+	fi
+}
+
+save_letsencryptkey() {
+	local keyfile=${pdir}/letsencrypt.key
+	printf -- \
+		"-----BEGIN RSA PRIVATE KEY-----\n%s\n-----END RSA PRIVATE KEY-----\n" \
+		${priv} | fold -w 65 | \
+		openssl rsa -inform pem -out $keyfile 2>/dev/null
+	if [ -e $keyfile ]; then
+		dc_log 7 "PrivateKey is valid and saved in $keyfile"
+	else
+		dc_log 4 "PrivateKey appers NOT to be valid"
+		exit 2
+	fi
+}
+
+read_domains() {
+	# Process the certificates for each of the domains in acme.json
+	case $acmeversion in
+		1) jq_filter='.Certificates[].Domain.Main' ;;
+		2) jq_filter='.[].Certificates[].domain.main' ;;
+	esac
+	domains=$(jq -r $jq_filter $acmefile)
+	if [ -n "$domains" ]; then
+		dc_log 7 "Extracting private key and cert bundle for domains $domains."
+	else
+		dc_log 4 "Unable to find any domains in $acmefile."
+		exit 2
+	fi
+}
+
+#
+# Traefik stores a cert bundle for each domain.  Within this cert
+# bundle there is both proper the certificate and the Let's Encrypt CA
+#
+save_certs() {
+	dc_log 5 "Extracting private keys and cert bundles in ${acmefile}"
+	case $acmeversion in
+		1)
+			jq_crtfilter='.Certificates[] | select (.Domain.Main == $domain )| .Certificate'
+			jq_keyfilter='.Certificates[] | select (.Domain.Main == $domain )| .Key'
+			;;
+		2)
+			jq_crtfilter='.[].Certificates[] | select (.domain.main == $domain )| .certificate'
+			jq_keyfilter='.[].Certificates[] | select (.domain.main == $domain )| .key'
+			;;
+	esac
+	for domain in $domains; do
+		crt=$(jq -e -r --arg domain "$domain" "$jq_crtfilter" $acmefile) || bad_acme
+		echo "${crt}" | ${CMD_DECODE_BASE64} > "${cdir}/${domain}.crt"
+		key=$(jq -e -r --arg domain "$domain" "$jq_keyfilter" $acmefile) || bad_acme
+		echo "${key}" | ${CMD_DECODE_BASE64} > "${pdir}/${domain}.key"
+	done
+}
+
+#
+# Run command in ACME_POSTHOOK if it contain a valid command and runsvdir is running.
+#
+run_posthook() {
+	if (pidof runsvdir >/dev/null && [ -n "$ACME_POSTHOOK" ] && command -v $ACME_POSTHOOK >/dev/null); then
+		local out=$(eval "$ACME_POSTHOOK" 2>&1)
+		[ -n "$out" ] && dc_log 7 "$ACME_POSTHOOK : $out"
+	fi
+}
+
+#
+# run
+#
+test_args $@
+test_dependencies
+test_acmefile
+test_certdir
+read_letsencryptkey
+make_certdirs
+save_letsencryptkey
+read_domains
+save_certs
+run_posthook
--- a/src/acme/entry.d/10-acme-common
+++ b/src/acme/entry.d/10-acme-common
@ -0,0 +1,58 @@
+#!/bin/sh
+#
+# 10-acme-common
+#
+# Define variables and functions used during container initialization.
+#
+# Variables defined in Dockerfile
+# DOCKER_ACME_SSL_DIR DOCKER_APPL_SSL_DIR
+#
+ACME_FILE=${ACME_FILE-/acme/acme.json}
+HOSTNAME=${HOSTNAME-$(hostname)}
+DOMAIN=${HOSTNAME#*.}
+DOCKER_APPL_SSL_CERT=${DOCKER_APPL_SSL_CERT-$DOCKER_APPL_SSL_DIR/cert.pem}
+DOCKER_APPL_SSL_KEY=${DOCKER_APPL_SSL_KEY-$DOCKER_APPL_SSL_DIR/priv_key.pem}
+DOCKER_ACME_SSL_H_CERT=$DOCKER_ACME_SSL_DIR/certs/${HOSTNAME}.crt
+DOCKER_ACME_SSL_H_KEY=$DOCKER_ACME_SSL_DIR/private/${HOSTNAME}.key
+DOCKER_ACME_SSL_D_CERT=$DOCKER_ACME_SSL_DIR/certs/${DOMAIN}.crt
+DOCKER_ACME_SSL_D_KEY=$DOCKER_ACME_SSL_DIR/private/${DOMAIN}.key
+
+#
+# Setup monitoring of ACME_FILE
+#
+acme_monitor_tls_cert() {
+	if (dc_is_installed jq && (dc_is_command inotifyd || dc_is_command inotifywait)); then
+		if [ -s $ACME_FILE ]; then
+			# run acme-extract.sh on cnt creation (and every time the json file changes)
+			dc_log 5 "Setup ACME TLS certificate monitoring"
+			if dc_is_command inotifyd; then
+				docker-service.sh "-n acme $(which inotifyd) $(which acme-extract.sh) $ACME_FILE:c"
+			else
+				docker-service.sh "-n acme sh -c \"while $(which inotifywait) -e close_write $ACME_FILE; do $(which acme-extract.sh); done\""
+			fi
+			# acme-extract.sh reports to logger but it is yet to be started so this run will be quiet
+			acme-extract.sh $ACME_FILE $DOCKER_ACME_SSL_DIR
+		fi
+	else
+		dc_log 5 "Not all required pkgs installed so cannot setup ACME TLS certificate monitoring"
+	fi
+}
+
+#
+# Arrange sym-links to support both host and domain certificates.
+#
+acme_symlink_tls_cert() {
+	if ([ -r $DOCKER_ACME_SSL_H_CERT ] && [ -r $DOCKER_ACME_SSL_H_KEY ]); then
+		dc_log 5 "Setting up ACME TLS certificate for host $HOSTNAME"
+		mkdir -p $DOCKER_APPL_SSL_DIR
+		ln -sf $DOCKER_ACME_SSL_H_CERT $DOCKER_APPL_SSL_CERT
+		ln -sf $DOCKER_ACME_SSL_H_KEY $DOCKER_APPL_SSL_KEY
+	else
+		if ([ -r $DOCKER_ACME_SSL_D_CERT ] && [ -r $DOCKER_ACME_SSL_D_KEY ]); then
+			dc_log 5 "Setting up ACME TLS certificate for domain $DOMAIN"
+			mkdir -p $DOCKER_APPL_SSL_DIR
+			ln -sf $DOCKER_ACME_SSL_D_CERT $DOCKER_APPL_SSL_CERT
+			ln -sf $DOCKER_ACME_SSL_D_KEY $DOCKER_APPL_SSL_KEY
+		fi
+	fi
+}
--- a/src/acme/entry.d/50-acme-monitor-tlscert
+++ b/src/acme/entry.d/50-acme-monitor-tlscert
@ -0,0 +1,13 @@
+#!/bin/sh
+#
+# 50-acme-monitor-tlscert
+#
+# Functions defined in:
+# 10-acme-common
+#
+
+#
+# Setup ACME monitor and arrange certificate symbolic links
+#
+acme_monitor_tls_cert
+acme_symlink_tls_cert
--- a/src/docker/bin/docker-common.sh
+++ b/src/docker/bin/docker-common.sh
@ -85,9 +85,25 @@ dc_log_stamp() {
 }

 #
-# Tests
+# Tests if command is in the path
 #
-dc_is_installed() { apk -e info $1 &>/dev/null ;} # true if pkg is installed
+dc_is_command() { [ -x "$(command -v $1)" ] ;}
+
+#
+# Tests if pkgs are installed
+#
+dc_is_installed() {
+	if dc_is_command apk; then
+		ver_cmd="apk -e info"
+	elif dc_is_command dpkg; then
+		ver_cmd="dpkg -s"
+	else
+		dc_log 5 "No package manager found among: apk dpkg"
+	fi
+	for cmd in $@; do
+		$ver_cmd $cmd > /dev/null 2>&1 || return 1
+	done
+}

 #
 # Update loglevel
@ -112,14 +128,14 @@ dc_pkg_versions() {
 	local kern=$(uname -r)
 	local host=$(uname -n)
 	dc_log 5 $host $os $kern
-	if [ -x "$(command -v apk)" ]; then
+	if dc_is_command apk; then
 		ver_cmd="apk info -s"
 		sed_flt="s/.*-(.*)-.*/\1/p"
-	elif [ -x "$(command -v apt)" ]; then
-		ver_cmd="apt list --installed"
-		sed_flt="s/[^ ]+ ([[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+).*/\1/p"
+	elif dc_is_command dpkg; then
+		ver_cmd="dpkg -s"
+		sed_flt="s/Version: ([[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+).*/\1/p"
 	else
-		dc_log 5 "No package manager found among: apk apt"
+		dc_log 5 "No package manager found among: apk dpkg"
 	fi
 	for pkg in $pkgs; do
 		ver=$($ver_cmd $pkg 2> /dev/null | sed -rn "$sed_flt")
--- a/src/docker/bin/docker-config.sh
+++ b/src/docker/bin/docker-config.sh
@ -11,6 +11,8 @@ HOSTNAME=${HOSTNAME-$(hostname)}
 DOMAIN=${HOSTNAME#*.}
 TLS_KEYBITS=${TLS_KEYBITS-2048}
 TLS_CERTDAYS=${TLS_CERTDAYS-30}
+DOCKER_CRONTAB_FILE=${DOCKER_CRONTAB_FILE-/etc/crontab}
+DOCKER_CRONTAB_ENV=${DOCKER_CRONTAB_ENV-CRONTAB_ENTRY}

 #
 # general file manipulation commands, used both during build and run time
@ -141,14 +143,34 @@ dc_persist_mvdirs() {

 #
 # Conditionally change owner of files.
+# -a all
+# -r readable
+# -w writable
+# -x executable
 #
-dc_chowncond() {
+dc_cond_chown() {
+	dc_log 7 "Called with args: $@"
+	OPTIND=1
+	local find_opts="! -perm -404"
+	while getopts ":arwx" opts; do
+		case "${opts}" in
+			a) find_opts="";;
+			r) find_opts="! -perm -404";;
+			w) find_opts="! -perm -606";;
+			x) find_opts="! -perm -505";;
+		esac
+	done
+	shift $((OPTIND -1))
 	local user=$1
-	local dir=$2
+	shift
 	if id $user > /dev/null 2>&1; then
-		if [ -n "$(find $dir ! -user $user -print -exec chown -h $user: {} \;)" ]; then
-			dc_log 5 "Changed owner to $user for some files in $dir"
-		fi
+		for dir in $@; do
+			if [ -n "$(find $dir ! -user $user $find_opts -print -exec chown -h $user: {} \;)" ]; then
+				dc_log 5 "Changed owner to $user for some files in $dir"
+			fi
+		done
+	else
+		dc_log 3 "User $user is unknown."
 	fi
 }

@ -212,6 +234,18 @@ dc_prune_pidfiles() {
 	done
 }

+#
+# Setup crontab entries
+#
+dc_crontab_entries() {
+	local entries="$(eval echo \${!$DOCKER_CRONTAB_ENV*})"
+	for entry in $entries; do
+		[ -z "${changed+x}" ] && local changed= && sed -i '/^\s*[0-9*]/d' $DOCKER_CRONTAB_FILE
+		echo "${!entry}" >> $DOCKER_CRONTAB_FILE
+		dc_log 5 "Added entry ${!entry} in $DOCKER_CRONTAB_FILE"
+	done
+}
+
 #
 # TLS/SSL Certificates [openssl]
 #
--- a/src/docker/bin/docker-runfunc.sh
+++ b/src/docker/bin/docker-runfunc.sh
@ -2,13 +2,14 @@
 #
 # docker-runfunc.sh
 #
-# Allow functions to be accessed from the commandline.
+# Allow functions to be accessed from the command line.
 #

 #
 # Source common functions.
 #
 . docker-common.sh
+. docker-config.sh

 #
 # dr_docker_call_func "$@"
@ -17,7 +18,7 @@ dr_docker_call_func() {
 	export DOCKER_RUNFUNC="$@"
 	local cmd=$1
 	shift
-	dc_log 7 "CMD:$cmd ARG:$@"
+#	dc_log 7 "CMD:$cmd ARG:$@"
 	$cmd "$@"
 	exit 0
 }
@ -28,7 +29,7 @@ dr_docker_call_func() {
 #
 dr_docker_run_parts() {
 	for file in $(find $1 -type f -name "$2" -executable 2>/dev/null|sort); do
-		dc_log 7 run_parts: executing $file
+#		dc_log 7 run_parts: executing $file
 		. $file
 	done
 }
--- a/src/docker/bin/docker-service.sh
+++ b/src/docker/bin/docker-service.sh
@ -61,6 +61,8 @@ init_service() {
 	local clearpid=
 	local sourcefile=
 	local sv_name cmd runsv_dir svlog_dir sv_log sv_down sv_force options
+	dc_log 7 "Called with args $@"
+	OPTIND=1
 	while getopts ":dfhln:s:q" opts; do
 		case "${opts}" in
 			d) sv_down="down"; add_opt "down";;
@ -104,6 +106,8 @@ init_service() {
 			!cat
 			chmod +x $runsv_dir/log/run
 		fi
+	else
+		dc_log 4 "Cannot find command."
 	fi
 	}

--- a/src/docker/config/docker-crontab
+++ b/src/docker/config/docker-crontab
@ -0,0 +1,7 @@
+# /etc/cron.d/docker-crontab: crontab entries for docker
+
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+
+# During initialization CRONTAB_ENTRY envvars will overwrite all cron commands
+# in this file. Cron commands start with * or digits with optional white space.
+
--- a/src/docker/entry.d/20-docker-print-versions
+++ b/src/docker/entry.d/20-docker-print-versions
@ -1,5 +1,5 @@
 #!/bin/sh
 #
-# 10-docker-print-versions
+# 20-docker-print-versions
 #
 dc_pkg_versions kopano-common kopano-webapp z-push-kopano
--- a/src/docker/entry.d/50-docker-crontab-entry
+++ b/src/docker/entry.d/50-docker-crontab-entry
@ -0,0 +1,15 @@
+#!/bin/sh
+#
+# 50-docker-crontab-entry
+#
+# Functions defined in:
+# docker-config.sh
+#
+#
+
+#
+# Add crontab entries if the config is unlocked.
+#
+if dc_is_unlocked; then
+	dc_crontab_entries
+fi
--- a/src/kopano/bin/kopano-webaddr.sh
+++ b/src/kopano/bin/kopano-webaddr.sh
@ -8,8 +8,8 @@ _webroot="https://download.kopano.io/community"
 _debroot="http://repo.z-hub.io/z-push:"
 _component="core"
 _stage="final"
-_dist="debian"
-_rel="9"
+_dist="ubuntu"
+_rel="20.04"
 _arch="amd64"

 #
--- a/src/kopano/entry.d/10-kopano-common
+++ b/src/kopano/entry.d/10-kopano-common
@ -5,88 +5,211 @@
 # Kopano now installs without any cfg files, so we just write custom values
 # into their target cfg file.
 #
+# Variables defined in Dockerfile
+# DOCKER_CONF_DIR1 DOCKER_CONF_DIR2 DOCKER_ACME_SSL_DIR DOCKER_APPL_SSL_DIR
+#

 #
 # Configuration
 #
-
 DOCKER_CONF_DIR1=${DOCKER_CONF_DIR1-/etc/kopano}
 DOCKER_CONF_DIR2=${DOCKER_CONF_DIR2-/usr/share/z-push}
-
-dagent_cfg_file=$DOCKER_CONF_DIR1/dagent.cfg
-gateway_cfg_file=$DOCKER_CONF_DIR1/gateway.cfg
-ical_cfg_file=$DOCKER_CONF_DIR1/ical.cfg
-ldap_cfg_file=$DOCKER_CONF_DIR1/ldap.cfg
-search_cfg_file=$DOCKER_CONF_DIR1/search.cfg
-server_cfg_file=$DOCKER_CONF_DIR1/server.cfg
-spamd_cfg_file=$DOCKER_CONF_DIR1/spamd.cfg
-spooler_cfg_file=$DOCKER_CONF_DIR1/spooler.cfg
-
+DOCKER_MAN5_DIR=${DOCKER_MAN5_DIR-/usr/share/man/man5/}
+DOCKER_APPL_SSL_CERT=${DOCKER_APPL_SSL_CERT-$DOCKER_APPL_SSL_DIR/cert.pem}
+DOCKER_APPL_SSL_KEY=${DOCKER_APPL_SSL_KEY-$DOCKER_APPL_SSL_DIR/priv_key.pem}
+DOCKER_LDAP_PMAP_FILE=${DOCKER_LDAP_PMAP_FILE-/usr/share/kopano/ldap.propmap.cfg}
+DOCKER_LDAP_SERVICES=${DOCKER_LDAP_SERVICES-archiver dagent gateway ical ldap search server spamd spooler}
 sqlstate_cfg_file=$DOCKER_CONF_DIR2/backend/sqlstatemachine/config.php
 zpush_cfg_file=$DOCKER_CONF_DIR2/config.php
+webapp_cfg_file=$DOCKER_CONF_DIR1/webapp/config.php
+webapp_mdm_cfg_file=$DOCKER_CONF_DIR1/webapp/config-mdm.php
+webapp_smime_cfg_file=$DOCKER_CONF_DIR1/webapp/config-smime.php
+

 #
-# Define environment variables
+# CLI commands
 #
-
-dagent_env_vars="LMTP_LISTEN LOG_LEVEL SPAM_HEADER_NAME SPAM_HEADER_VALUE"
-gateway_env_vars="BYPASS_AUTH DISABLE_PLAINTEXT_AUTH HTML_SAFETY_FILTER IMAP_CAPABILITY_IDLE IMAP_EXPUNGE_ON_DELETE IMAP_IGNORE_COMMAND_IDLE IMAP_LISTEN IMAP_MAX_FAIL_COMMANDS IMAP_MAX_MESSAGESIZE IMAP_ONLY_MAILFOLDERS IMAP_PUBLIC_FOLDERS IMAPS_LISTEN POP3_LISTEN POP3S_LISTEN DISABLE_PLAINTEXT_AUTH"
-ical_env_vars="ICAL_LISTEN ICALS_LISTEN ENABLE_ICAL_GET"
-ldap_env_vars="LDAP_URI LDAP_HOST LDAP_PORT LDAP_PROTOCOL LDAP_SEARCH_BASE LDAP_USER_TYPE_ATTRIBUTE_VALUE LDAP_GROUP_TYPE_ATTRIBUTE_VALUE LDAP_USER_SEARCH_FILTER"
-search_env_vars="INDEX_PROCESSES INDEX_DRAFTS INDEX_JUNK SUGGESTIONS INDEX_ATTACHMENTS INDEX_ATTACHMENT_MAX_SIZE"
-server_env_vars="MYSQL_HOST MYSQL_PORT MYSQL_DATABASE MYSQL_USER MYSQL_PASSWORD DISABLED_FEATURES USER_PLUGIN LOG_LEVEL SEARCH_TIMEOUT"
-spamd_env_vars="LOG_METHOD SPAM_DIR SPAM_DB HEADER_TAG LEARN_HAM HAM_DIR SA_GROUP"
-spooler_env_vars="SMTP_SERVER SMTP_PORT LOG_LEVEL"
-
-sqlstate_env_vars="STATE_SQL_ENGINE STATE_SQL_SERVER STATE_SQL_PORT STATE_SQL_DATABASE STATE_SQL_USER STATE_SQL_PASSWORD STATE_SQL_OPTIONS"
-zpush_env_vars="TIMEZONE USE_CUSTOM_REMOTE_IP_HEADER USE_FULLEMAIL_FOR_LOGIN STATE_MACHINE STATE_DIR LOGBACKEND LOGLEVEL LOGAUTHFAIL LOG_SYSLOG_PROGRAM LOG_SYSLOG_FACILITY SYNC_CONFLICT_DEFAULT PING_INTERVAL FILEAS_ORDER SYNC_MAX_ITEMS UNSET_UNDEFINED_PROPERTIES ALLOW_WEBSERVICE_USERS_ACCESS USE_PARTIAL_FOLDERSYNC"
-
-#
-# Define helpers
-#
-
-_kopano_apply_envvars_gen() {
-	# move existing cfg files
-	local cfg_file=$1
-	shift
-	local env_vars=$@
-	if [ -e $cfg_file ]; then
-		mv -f $cfg_file $cfg_file.orig
-	fi
-	for env_var in $env_vars; do
-		if [ -n "${!env_var+x}" ]; then
-			dc_log 5 "Setting ${env_var,,} = ${!env_var} in $cfg_file"
-			echo ${env_var,,} = ${!env_var} >> $cfg_file
-		fi
+list_parms() {
+	local services="$DOCKER_LDAP_SERVICES $DAGENT_PLUGINS"
+	[ $# -ge 0 ] && services="$@"
+	for service in $services; do
+		[ $# -ne 1 ] && echo "# $service"
+		local man_file=$(kopano_gen_filename_man $service)
+		kopano_get_envvars_man $man_file
+		kopano_get_envvars_ext $service
+		[ $# -ne 1 ] && echo
 	done
 }

-_kopano_apply_phpenvvars_gen() {
-	local cfg_file=$1
-	shift
-	local env_vars=$@
+#
+# Apply environment variables to configuration files.
+# Use all valid keys (variables) for a service to see if there is a envvar with
+# identical name, if so apply its value to the config file.
+# With kopano-core use man page files to lookup valid keys.
+# With kopano-webapp and z-push use installed config file to find valid keys.
+#
+kopano_apply_envvars_core() {
+	kopano_enable_envvars_plugin
+	for service in $DOCKER_LDAP_SERVICES $DAGENT_PLUGINS; do
+		kopano_apply_envvars_cfg $service
+	done
+}
+
+kopano_apply_envvars_webapp() {
+	kopano_apply_envvars_php $webapp_cfg_file
+	kopano_apply_envvars_php $webapp_mdm_cfg_file
+	kopano_apply_envvars_php $webapp_smime_cfg_file
+}
+
+kopano_apply_envvars_zpush() {
+	kopano_apply_envvars_php $sqlstate_cfg_file
+	kopano_apply_envvars_php $zpush_cfg_file
+}
+
+#
+# kopano_apply_envvars_cfg <service>
+#
+kopano_apply_envvars_cfg() {
+	local service=$1
+	local cfg_file=$(kopano_gen_filename_cfg $service)
+	local man_file=$(kopano_gen_filename_man $service)
+	local env_vars="$(kopano_get_envvars_ext $service)"
+	if [ -f $man_file ]; then
+		env_vars="$(kopano_get_envvars_man $man_file) $env_vars"
+	else
+		dc_log 6 "Could not find $man_file"
+	fi
 	if [ -e $cfg_file ]; then
+		mv -f $cfg_file $cfg_file.bak
+	fi
+	for env_var in $env_vars; do
+		kopano_set_envvars_cfg $service $env_var
+	done
+}
+
+kopano_apply_envvars_php() {
+	local cfg_file=$1
+	if [ -e $cfg_file ]; then
+		local env_vars="$(kopano_get_envvars_php $cfg_file)"
 		for env_var in $env_vars; do
 			if [ -n "${!env_var}" ]; then
+				[ -z "${changed+x}" ] && local changed= && cp -f $cfg_file $cfg_file.bak
 				dc_log 5 "Setting ${env_var} = ${!env_var} in $cfg_file"
-				sed -ri "s/(\s*define).+${env_var}.+/\1\(\x27${env_var}\x27, \x27${!env_var}\x27\);/g" $cfg_file
+				sed -ri "s/(\s*define[('\"]+${env_var}['\",]+).+/\1 ${!env_var});/Ig" $cfg_file
 			fi
 		done
 	fi
 }

-kopano_apply_envvars() {
-	_kopano_apply_envvars_gen $dagent_cfg_file   $dagent_env_vars
-	_kopano_apply_envvars_gen $gateway_cfg_file  $gateway_env_vars
-	_kopano_apply_envvars_gen $ical_cfg_file     $ical_env_vars
-	_kopano_apply_envvars_gen $ldap_cfg_file     $ldap_env_vars
-	_kopano_apply_envvars_gen $search_cfg_file   $search_env_vars
-	_kopano_apply_envvars_gen $server_cfg_file   $server_env_vars
-	_kopano_apply_envvars_gen $spamd_cfg_file    $spamd_env_vars
-	_kopano_apply_envvars_gen $spooler_cfg_file  $spooler_env_vars
+#
+# kopano_set_envvars_cfg <service> <envvar name> [explicit parameter name]
+#
+kopano_set_envvars_cfg() {
+	local service=$1
+	local cfg_file=$(kopano_gen_filename_cfg $service)
+	local env_var=$2
+	local uniq_var=${service^^}_${env_var}
+	local cfg_par="${3-$env_var =}"
+	if [ -n "${!uniq_var+x}" ]; then
+		local env_val="${!uniq_var}"
+	elif [ -n "${!env_var+x}" ]; then
+		local env_val="${!env_var}"
+	fi
+	if [ -n "${env_val+x}" ]; then
+		dc_log 5 "Setting ${cfg_par,,} ${env_val} in $cfg_file"
+		echo ${cfg_par,,} ${env_val} >> $cfg_file
+	fi
 }

-kopano_apply_phpenvvars() {
-	_kopano_apply_phpenvvars_gen $sqlstate_cfg_file $sqlstate_env_vars
-	_kopano_apply_phpenvvars_gen $zpush_cfg_file    $zpush_env_vars
+#
+# External parameters
+# services without a man file or additional parameters are listed here.
+#
+kopano_get_envvars_ext() {
+	local service=$1
+	local vars
+	case $service in
+		movetopublicldap)
+			vars="ldap_public_folder_attribute ldap_public_folder_attribute_token"
+			;;
+	esac
+	echo $vars | tr "[:lower:]" "[:upper:]" | tr " " "\n" | sort -u
+}
+
+#
+# Python plugins
+#
+kopano_enable_envvars_plugin() {
+	for service in dagent spooler; do
+		local envvar=${service^^}_PLUGINS
+		for plugin in ${!envvar}; do
+			local pyfile=$(find /usr/share/kopano* -name ${plugin}.py)
+			if [ -z "$pyfile" ]; then
+				dc_log 5 "Unable to locate ${service} python plugin ${plugin}"
+			else
+				dc_log 5 "Enabling ${service} python plugin ${plugin}"
+				mkdir -p $DOCKER_CONF_DIR1/${service}
+				ln -sf ${pyfile} $DOCKER_CONF_DIR1/${service}
+				export ${service^^}_PLUGIN_PATH=$DOCKER_CONF_DIR1/${service}
+				export ${service^^}_PLUGIN_ENABLED=yes
+			fi
+		done
+	done
+}
+
+#
+# Helpers
+#
+kopano_gen_filename_man() { echo $DOCKER_MAN5_DIR/kopano-$1.cfg.5.gz ;}
+kopano_gen_filename_cfg() { echo $DOCKER_CONF_DIR1/$1.cfg ;}
+kopano_get_envvars_man() { zcat $1 | sed -r "/^\.SS/!d;{s/^\.SS (.*)/\U\1/g;s/,//g}" | sort -u ;}
+kopano_get_envvars_php() { sed -nr "/define\(/Is/.*define\(['\"](.*)['\"], .*/\1/Ip" $1 | sort -u ;}
+
+#
+# Update SSL_CERTIFICATE_FILE and SSL_PRIVATE_KEY_FILE.
+#
+kopano_export_tls_cert() {
+	if ([ -f "$DOCKER_APPL_SSL_CERT" ] && [ -f "$DOCKER_APPL_SSL_KEY" ]); then
+		export SSL_CERTIFICATE_FILE=${SSL_CERTIFICATE_FILE-$DOCKER_APPL_SSL_CERT}
+		export SSL_PRIVATE_KEY_FILE=${SSL_PRIVATE_KEY_FILE-$DOCKER_APPL_SSL_KEY}
+	fi
+	if ([ -f "$SSL_CERTIFICATE_FILE" ] && [ -f "$SSL_PRIVATE_KEY_FILE" ]); then
+		dc_cond_chown $DOCKER_APPL_RUNAS $SSL_CERTIFICATE_FILE $SSL_PRIVATE_KEY_FILE
+	fi
+}
+
+#
+# Generate self signed certificate if any of IMAPS_LISTEN, POP3S_LISTEN or
+# ICALS_LISTEN is no-empty but no certificates are given.
+#
+kopano_generate_tls_cert() {
+	for prot in IMAPS_LISTEN POP3S_LISTEN ICALS_LISTEN; do
+		if [ -n "${!prot}" ]; then
+			local secure="$prot=${!prot}"
+			break
+		fi
+	done
+	if ([ -z "$SSL_CERTIFICATE_FILE" ] && [ -n "$secure" ] && dc_is_installed openssl); then
+		dc_log 4 "$secure, but no certs given, so generating self-signed cert for host $HOSTNAME"
+		mkdir -p $DOCKER_APPL_SSL_DIR
+		dc_tls_setup_selfsigned_cert $DOCKER_APPL_SSL_CERT $DOCKER_APPL_SSL_KEY
+	fi
+}
+
+#
+# LDAP directives
+#
+kopano_apply_envvar_propmap() {
+	local service=ldap
+	local cfg_file=$(kopano_gen_filename_cfg $service)
+	local env_var=LDAP_PROPMAP
+	if [ -n "${!env_var+x}" ]; then
+		if [ -z "${!env_var}" ]; then
+			typeset "$env_var=$DOCKER_LDAP_PMAP_FILE"
+		elif [ ! -e "${!env_var}" ]; then
+			dc_log 5 "No ${!env_var} so copying $DOCKER_LDAP_PMAP_FILE there."
+			cp -f $DOCKER_LDAP_PMAP_FILE ${!env_var}
+		fi
+		kopano_set_envvars_cfg $service $env_var '!propmap'
+	fi
 }
--- a/src/kopano/entry.d/30-kopano-migrate
+++ b/src/kopano/entry.d/30-kopano-migrate
@ -0,0 +1,34 @@
+#!/bin/sh
+#
+# 30-kopano-migrate
+#
+# Try to make configs compatible with new version if MIGRATE_CONFIG is defined.
+# Set MIGRATE_CONFIG=1 2 3 to list of fixes or MIGRATE_CONFIG=all to attempt all fixes.
+#
+kopano_apply_migrate_fixes() {
+	local applied
+	if [ -n "$MIGRATE_CONFIG" ]; then
+		for fix in ${MIGRATE_CONFIG/all/1 2}; do # list all fixes here
+		case $fix in
+		1) dc_replace /etc/kopano/webapp/config.php 'define("INSECURE_COOKIES", true);' 'define("SECURE_COOKIES", false);' ;;
+		2)
+			for plugin in smime mdm; do
+				local cfg_file=$DOCKER_CONF_DIR1/webapp/config-$plugin.php
+				local src_file=$DOCKER_SMPL_DIR1/webapp/config-$plugin.php
+				[ ! -f "$cfg_file" ] && cp "$src_file" "$cfg_file"
+			done
+			;;
+		*) fix= ;;
+		esac
+		if [ -n "$fix" ]; then
+			applied="$applied $fix"
+		fi
+		done
+		dc_log 5 "Applied fixes;$applied to configuration since MIGRATE_CONFIG=$MIGRATE_CONFIG"
+	fi
+}
+
+#
+# run
+#
+kopano_apply_migrate_fixes
--- a/src/kopano/entry.d/50-kopano-config
+++ b/src/kopano/entry.d/50-kopano-config
@ -11,6 +11,11 @@
 # Configure Kopano if it is unlocked.
 #
 if dc_is_unlocked; then
-	kopano_apply_envvars
-	kopano_apply_phpenvvars
+	kopano_export_tls_cert
+	kopano_generate_tls_cert
+	kopano_export_tls_cert
+	kopano_apply_envvars_core
+	kopano_apply_envvars_webapp
+	kopano_apply_envvars_zpush
+	kopano_apply_envvar_propmap
 fi
--- a/src/kopano/plugin/movetopublicldap.cfg
+++ b/src/kopano/plugin/movetopublicldap.cfg
@ -0,0 +1,32 @@
+# movetopublicldap.cfg
+#
+# Move mail to public ldap configuration
+#
+# This configuration file is used by the dagent plugin 'movetopublicldap.py'
+# The default location is '/etc/kopano/movetopublicldap.cfg'
+#
+
+#
+# The plugin reads two coniguration files.
+# First it looks for parameters in '/etc/kopano/ldap.cfg' allowing
+# common conifuration parameters to be kept in on place.
+# Second it reads '/etc/kopano/movetopublicldap.cfg'
+#
+
+#
+# Parameters that can be set in both files:
+#
+# ldap_uri =
+# ldap_starttls = no
+# ldap_search_base =
+# ldap_bind_user =
+# ldap_bind_passwd =
+# ldap_user_unique_attribute = uid
+#
+
+#
+# Parameters that can only be set in ovetopublicldap.cfg:
+#
+# ldap_public_folder_attribute = kopanoResourceType
+# ldap_public_folder_attribute_token = publicFolder
+#
--- a/src/kopano/plugin/movetopublicldap.py
+++ b/src/kopano/plugin/movetopublicldap.py
@ -0,0 +1,229 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+""" (c) 2019 Kopano
+
+movetopublicldap.py
+
+This is an LDAP lookup extension to the move to public plugin movetopublic.py.
+The changes to the original work are highlihgted below.
+
+The move to public plugin moves incoming messages to a folder in the public
+store. If folders are missing they will be created.
+
+A LDAP entry including:
+
+kopanoAccount: 1
+kopanoResourceType: publicFolder:<public folder>
+
+will have its email delivered to the public store in <public folder>.
+The token match is case sensitive and there must be a colon ':' separating
+the token and the public folder name. The folder name can contain space and
+sub folders, which are distinguished using forward slash '/'.
+So if we have 'kopanoResourceType: publicFolder:Public Stores/public' emails will
+be delivered to 'Public Folders/Public Stores/public'.
+
+The parameters in /etc/kopano/ldap.cfg will be used for the LDAP query.
+The LDAP attribute holding the token and the token itself have the following
+default values, which can be modified in /etc/kopano/movetopublicldap.cfg
+if desired.
+
+ldap_public_folder_attribute = kopanoResourceType
+ldap_public_folder_attribute_token = publicFolder
+
+"""
+from sys import hexversion
+from MAPI.Util import GetPublicStore
+from MAPI.Struct import NEWMAIL_NOTIFICATION
+from MAPI import MAPI_UNICODE, MAPI_MODIFY, OPEN_IF_EXISTS, MDB_WRITE
+from MAPI.Tags import (PR_RECEIVED_BY_EMAIL_ADDRESS_W, PR_EC_COMPANY_NAME_W,
+	PR_IPM_PUBLIC_FOLDERS_ENTRYID, PR_ENTRYID, PR_MAILBOX_OWNER_ENTRYID,
+	IID_IMessage, IID_IExchangeManageStore)
+from plugintemplates import IMapiDAgentPlugin, MP_CONTINUE, MP_STOP_SUCCESS
+from zconfig import ZConfigParser
+import configparser
+import ldap
+import os.path
+
+class KConfigParser(ZConfigParser):
+	""" Extends zconfig.ZConfigParser to also allow !directive in cfg files.
+	Change type addition. """
+	def __init__(self, configfile, defaultconfig={}):
+		self.config = configparser.ConfigParser(defaults=defaultconfig,
+			delimiters=('='), comment_prefixes=('#', '!'), allow_no_value=True)
+		self.readZConfig(configfile)
+
+class MoveToPublic(IMapiDAgentPlugin):
+
+	prioPreDelivery = 50
+
+	config = {}
+
+	CONFIGFILES = ['/etc/kopano/ldap.cfg', '/etc/kopano/movetopublicldap.cfg']
+
+	DEFAULTCONFIG = {
+		'ldap_uri': None,
+		'ldap_starttls': "no",
+		'ldap_search_base': None,
+		'ldap_bind_user': None,
+		'ldap_bind_passwd': None,
+		'ldap_user_unique_attribute': "uid",
+		'ldap_public_folder_attribute': "kopanoResourceType",
+		'ldap_public_folder_attribute_token': "publicFolder"
+	}
+
+	def __init__(self, logger):
+		IMapiDAgentPlugin.__init__(self, logger)
+		self.readconfig(self.CONFIGFILES, self.DEFAULTCONFIG)
+
+	def readconfig(self, configfiles=CONFIGFILES, defaultconfig=DEFAULTCONFIG):
+		""" Reads ldap.cfg and movetopublicldap.cfg into self.config.
+		Change type addition. """
+		options = [opt.split('_', 1)[1] for opt in defaultconfig.keys()]
+		config = None
+		for configfile in configfiles:
+			if os.path.isfile(configfile):
+				self.logger.logDebug("*--- Reading file {}".format(configfile))
+				if not config:
+					config = KConfigParser(configfile, defaultconfig)
+				else:
+					config = KConfigParser(configfile, config.options())
+			else:
+				self.logger.logDebug("*--- Can't find file {}".format(configfile))
+		self.config = config.getdict('ldap',options)
+		self.logger.logDebug("*--- Config list {}".format(self.config))
+		return self.config
+
+	def searchfilter(self, recipient):
+		""" (&(uid=recipient)(kopanoResourceType=publicFolder:*)).
+		Change type addition. """
+		return ("(&({}={})({}={}:*))"
+			.format(self.config['user_unique_attribute'],
+			recipient,
+			self.config['public_folder_attribute'],
+			self.config['public_folder_attribute_token']))
+
+	def searchquery(self, recipient):
+		""" Query a LDAP/AD driectory server to lookup recipient using
+		search_base and return public_folder_attribute.
+		Change type addition. """
+		if not self.config or not self.config['uri']:
+			self.logger.logError(("!--- ldap_uri is not defined."
+				" Please check {}" .format(self.CONFIGFILES)))
+			return None
+		else:
+			l = ldap.initialize(self.config['uri'])
+			try:
+				l.protocol_version = ldap.VERSION3
+				if self.config['starttls'] == 'yes':
+					l.start_tls_s()
+				l.simple_bind_s(self.config['bind_user'] or u'', \
+					self.config['bind_passwd'] or u'')
+			except ldap.SERVER_DOWN as e:
+				self.logger.logError(("!--- LDAP server is not reachable {}"
+				.format(e)))
+				return None
+			except ldap.INVALID_CREDENTIALS as e:
+				self.logger.logError(("!--- Invalid LDAP credentials {}"
+					" Please check {}" .format(e, self.CONFIGFILES)))
+				l.unbind_s()
+				return None
+			except ldap.LDAPError as e:
+				self.logger.logError("!--- LDAPError {}".format(e))
+				l.unbind_s()
+				return None
+			try:
+				result = l.search_s(self.config['search_base'], \
+					ldap.SCOPE_SUBTREE, self.searchfilter(recipient), \
+					[self.config['public_folder_attribute']])
+			except ldap.LDAPError as e:
+				self.logger.logError("!--- LDAPError {}".format(e))
+			l.unbind_s()
+		return result
+
+	def publicfolder(self, recipient):
+		""" Check for ldap_public_folder_attribute_token and return folder.
+		Change type addition. """
+		destination_folder = []
+		result = self.searchquery(recipient)
+		if result:
+			tokenandfolder = (result[0][1]
+				.get(self.config['public_folder_attribute'])[0].decode('utf-8'))
+			if tokenandfolder:
+				destination_folder = tokenandfolder.split(':')[1]
+				if destination_folder:
+					self.logger.logDebug(("*--- Found public folder {}"
+						"for recipient {}".format(
+						destination_folder.encode('utf-8'),
+						recipient.encode('utf-8'))))
+		return destination_folder
+
+	def PreDelivery(self, session, addrbook, store, folder, message):
+		""" Original code from movetopublic.py with call to self.publicfolder().
+		Change type modification. """
+
+		props = message.GetProps([PR_RECEIVED_BY_EMAIL_ADDRESS_W], 0)
+		if props[0].ulPropTag != PR_RECEIVED_BY_EMAIL_ADDRESS_W:
+			self.logger.logError("!--- Not received by emailaddress")
+			return MP_CONTINUE,
+
+		recipient = props[0].Value.lower()
+		if not recipient:
+			self.logger.logError("!--- No recipient in props {}".format(props))
+			return MP_CONTINUE,
+
+		recipfolder = self.publicfolder(recipient)
+		if not recipfolder:
+			self.logger.logDebug(("*--- No public folder for recipient {}"
+				.format(recipient.encode('utf-8'))))
+			return MP_CONTINUE,
+
+		publicstore = GetPublicStore(session)
+		if not publicstore:
+			storeprops = store.GetProps([PR_MAILBOX_OWNER_ENTRYID], 0)
+			if storeprops[0].ulPropTag == PR_MAILBOX_OWNER_ENTRYID:
+				user = addrbook.OpenEntry(storeprops[0].Value, None, 0)
+				userprops = user.GetProps([PR_EC_COMPANY_NAME_W], 0)
+				if userprops[0].ulPropTag == PR_EC_COMPANY_NAME_W:
+					companyname = userprops[0].Value
+				else:
+					companyname = None
+
+				if not companyname:
+					self.logger.logError(("!--- Cannot open a public store."
+						' Use "kopano-storeadm -P"'
+						" to create one if it is missing."))
+					return MP_CONTINUE,
+
+				ema = store.QueryInterface(IID_IExchangeManageStore)
+				publicstoreid = ema.CreateStoreEntryID(None, companyname, MAPI_UNICODE)
+				publicstore = session.OpenMsgStore(0, publicstoreid, None, MDB_WRITE)
+
+		publicfolders = publicstore.OpenEntry(
+			publicstore.GetProps([PR_IPM_PUBLIC_FOLDERS_ENTRYID], 0)[0].Value,
+			None, MAPI_MODIFY)
+
+		folderlist = recipfolder.split('/')
+		folder = publicfolders
+		for foldername in folderlist:
+			if len(foldername) > 0:
+				if hexversion >= 0x03000000:
+					folder = folder.CreateFolder(0, foldername,
+						"Create by Move to Public plugin", None,
+						OPEN_IF_EXISTS | MAPI_UNICODE)
+				else:
+					folder = folder.CreateFolder(0, foldername,
+						"Create by Move to Public plugin", None, OPEN_IF_EXISTS)
+
+		msgnew = folder.CreateMessage(None, 0)
+		tags = message.GetPropList(MAPI_UNICODE)
+		message.CopyProps(tags, 0, None, IID_IMessage, msgnew, 0)
+
+		msgnew.SaveChanges(0)
+		folderid = folder.GetProps([PR_ENTRYID], 0)[0].Value
+		msgid = msgnew.GetProps([PR_ENTRYID], 0)[0].Value
+
+		publicstore.NotifyNewMail(NEWMAIL_NOTIFICATION(msgid, folderid, 0, None, 0))
+
+		self.logger.logInfo(("*--- Message moved to public folder {}"
+			.format(recipfolder)))
+
+		return MP_STOP_SUCCESS,
--- a/src/notused/plugin/ldaptocfg.py
+++ b/src/notused/plugin/ldaptocfg.py
@ -0,0 +1,97 @@
+"""
+This code will query a LDAP/AD driectory server and updated the cgf file
+/etc/kopano/movetopublic.cfg
+/usr/share/kopano-dagent/python/plugins/movetopublic.cfg
+
+export PYTHONPATH=/usr/share/kopano-dagent/python
+
+"""
+from zconfig import ZConfigParser
+import configparser
+import ldap
+
+class KConfigParser(ZConfigParser):
+	""" allow !directive in cfg files """
+	def __init__(self, configfile, defaultconfig={}):
+		self.config = configparser.ConfigParser(defaults=defaultconfig, \
+			delimiters=('='), comment_prefixes=('#', '!'))
+		self.readZConfig(configfile)
+
+class ldapstores():
+	defaultconfig = {
+		'ldap_uri': None,
+		'ldap_search_base': None,
+		'ldap_bind_user': None,
+		'ldap_bind_passwd': None,
+		'ldap_user_unique_attribute': "uid",
+		'ldap_user_search_filter': "(kopanoAccount=1)",
+#		'ldap_user_search_filter': "(&(kopanoAccount=1)(kopanoResourceType=publicFolder:*))",
+		'ldap_public_folder_attribute': "kopanoResourceType",
+		'ldap_public_folder_attribute_token': "publicFolder"
+	}
+
+	def __init__(self, configfile = '/etc/kopano/ldap.cfg'):
+		self.readconfig(configfile)
+
+	def readconfig(self, configfile):
+		config = KConfigParser(configfile, self.defaultconfig)
+		options = [opt.split('_', 1)[1] for opt in self.defaultconfig.keys()]
+		self.config = config.getdict('ldap',options)
+		return self.config
+
+	def searchquery(self):
+		if (self.config['uri'] is None):
+			print ("ldap_uri is None")
+			sys.exit(0)
+		else:
+			l = ldap.initialize(self.config['uri'])
+			try:
+				l.protocol_version = ldap.VERSION3
+				l.simple_bind_s(self.config['bind_user'] or u'', \
+					self.config['bind_passwd'] or u'')
+			except ldap.INVALID_CREDENTIALS:
+				sys.exit(0)
+			except ldap.LDAPError as e:
+				print (e)
+				sys.exit(0)
+			try:
+				ldap_result_id = l.search(self.config['search_base'], \
+					ldap.SCOPE_SUBTREE, self.config['user_search_filter'], \
+					[self.config['user_unique_attribute'], \
+					self.config['public_folder_attribute']])
+				results = []
+				while 1:
+					result_type, result_data = l.result(ldap_result_id, 0)
+					if (result_data == []):
+						break
+					else:
+						if result_type == ldap.RES_SEARCH_ENTRY:
+							results.append(result_data[0])
+			except ldap.LDAPError as e:
+				print (e)
+			l.unbind_s()
+		return results
+
+	def findpublic(self):
+		stores = self.searchquery()
+		public = []
+		for store in stores:
+			recipient = store[1].get(self.config['user_unique_attribute'])
+			tokenandfolder = store[1].get(self.config['ldap_public_folder_attribute'])
+			if tokenandfolder:
+				token = tokenandfolder.split(':')[0]
+				destination_folder = tokenandfolder.split(':')[1]
+				if (token == self.config['ldap_public_folder_attribute_token']);
+					public[recipient] = destination_folder
+		return public
+
+	def printpublic(self, outputfile = '/etc/kopano/movetopublic.cfg'):
+		public = self.findpublic()
+		i = 1
+		for recipient in public.keys():
+			print ("rule%d_recipient = %s", i, recipient)
+			print ("rule%d_destination_folder = %s", i, public[recipient])
+			i += 1
+
+s = ldapstores()
+s.printpublic()
--- a/test/Makefile
+++ b/test/Makefile
@ -0,0 +1,318 @@
+# Makefile
+#
+# test
+#
+
+-include    *.mk
+
+TST_REPO ?= mlan/kopano
+TST_VER  ?= latest
+_ver      = $(if $(findstring latest,$(1)),$(2),$(2)-$(1))
+
+NET_NAME ?= test-net
+NET_ENV  ?= --network $(NET_NAME)
+
+AD_BASE  ?= dc=example,dc=com
+AD_DOM   ?= $(call ad_sub_dot, $(AD_BASE))
+AD_DC    ?= $(call ad_cut_dot, 1, 1, $(AD_DOM))
+AD_GRP_OU ?= groups
+AD_USR_OB ?= kopano-user
+AD_USR_OU ?= users
+AD_USR_CN ?= hero
+AD_USR_PW ?= enigma
+
+MAIL_SUB ?= ~~~test~message~~~
+MAIL_MSG ?= Enjoy!
+
+WEB_TIT  ?= "Kopano WebApp"
+
+SQL_BASE ?= kopano
+SQL_USER ?= kopano
+SQL_PASS ?= secret
+SQL_ENV  ?= \
+-e MYSQL_DATABASE=$(SQL_BASE) \
+-e MYSQL_USER=$(SQL_USER) \
+-e MYSQL_PASSWORD=$(SQL_PASS) \
+
+APP_NAME ?= app
+APP_FQDN ?= $(APP_NAME).$(AD_DOM)
+APP_CERT ?= ssl/$(APP_FQDN).crt
+APP_KEY  ?= ssl/$(APP_FQDN).key
+APP_VOL  ?=
+APP_SLOG ?= 7
+APP_ALOG ?= 6
+APP_ENV  ?= $(NET_ENV) $(SQL_ENV) \
+--name $(APP_NAME) \
+--hostname $(APP_FQDN) \
+-e MYSQL_HOST=$(DB_NAME) \
+-e USER_PLUGIN=ldap \
+-e LDAP_URI=ldap://$(AUT_NAME):389/ \
+-e LDAP_SEARCH_BASE=$(AD_BASE) \
+-e LDAP_USER_TYPE_ATTRIBUTE_VALUE=$(AD_USR_OB) \
+-e LDAP_GROUP_TYPE_ATTRIBUTE_VALUE=$(AD_GRP_OU) \
+-e LDAP_PROPMAP= \
+-e IMAP_LISTEN=*:143 \
+-e POP3_LISTEN=*:110 \
+-e ICAL_LISTEN=*:8080 \
+-e DISABLED_FEATURES= \
+-e LOGLEVEL=LOGLEVEL_DEBUG \
+-e SYSLOG_LEVEL=$(APP_SLOG) \
+-e LOG_LEVEL=$(APP_ALOG)
+APPS_ENV ?= $(APP_ENV) \
+-e IMAPS_LISTEN=*:993 \
+-e POP3S_LISTEN=*:995 \
+-e ICALS_LISTEN=*:8443
+APPA_ENV ?= $(APPS_ENV) \
+-v $(shell pwd)/acme:/acme
+
+AUT_NAME ?= auth
+AUT_IMG  ?= mlan/openldap
+AUT_FQDN ?= $(AUT_NAME).$(AD_DOM)
+AUT_VOL  ?=
+AUT_ENV  ?= $(NET_ENV) \
+--name $(AUT_NAME) \
+--hostname $(AUT_FQDN)
+
+TAW_NAME ?= throwaway
+TAW_ENV  ?= --entrypoint /bin/bash \
+--name $(TAW_NAME) \
+--rm
+
+CURL_OPT ?= -s -v
+CURL_IMG ?= curlimages/curl
+CURL_ENV ?= $(NET_ENV) \
+-i --rm
+
+DB_NAME  ?= db
+DB_IMG   ?= mariadb
+DB_FQDN  ?= $(DB_NAME).$(AD_DOM)
+DB_VOL   ?=
+DB_CMD   ?= --log_warnings=1
+DB_ENV   ?= $(NET_ENV) $(SQL_ENV) \
+--name $(DB_NAME) \
+--hostname $(DB_FQDN) \
+-e MYSQL_ROOT_PASSWORD=$(SQL_PASS)
+
+GREP_ENV ?=
+
+TST_W8S1 ?= 1
+TST_W8S2 ?= 20
+TST_W8L1 ?= 20
+TST_W8L2 ?= 120
+
+export define LDIF_ADD_DATA
+dn: $(AD_BASE)
+objectClass: organization
+objectClass: dcObject
+dc: $(AD_DC)
+o: $(AD_DOM)
+
+dn: ou=$(AD_USR_OU),$(AD_BASE)
+objectClass: organizationalUnit
+ou: $(AD_USR_OU)
+
+dn: ou=$(AD_GRP_OU),$(AD_BASE)
+objectClass: organizationalUnit
+ou: $(AD_GRP_OU)
+
+dn: uid=$(AD_USR_CN),ou=$(AD_USR_OU),$(AD_BASE)
+objectClass: inetOrgPerson
+objectClass: $(AD_USR_OB)
+cn: $(AD_USR_CN)
+sn: $(AD_USR_CN)
+uid: $(AD_USR_CN)
+mail: $(AD_USR_CN)@$(AD_DOM)
+kopanoAccount: 1
+userPassword: $(AD_USR_PW)
+endef
+
+
+variables:
+	make -pn | grep -A1 "^# makefile"| grep -v "^#\|^--" | sort | uniq
+
+ps:
+	docker ps -a
+
+test-all: test-up_0 test_1 test_2 test_3
+	
+
+test_%: test-up_% test-waitl_% test-logs_% test-service_% test-down_%
+	
+
+test-pull:
+	$(addprefix docker pull ,$(CURL_IMG); $(AUT_IMG); $(DB_IMG))
+
+test-up_0: test-up-net
+	#
+	#
+	#
+	# test (0) run without envvars (is there smoke?)
+	#
+	# run containers see if there are logs and stop.
+	#
+	#
+	docker run -d --name $(APP_NAME) $(TST_REPO):$(call _ver,$(TST_VER),core)
+	sleep $(TST_W8L1)
+	docker container logs $(APP_NAME) | grep 'docker-entrypoint.sh'
+	docker rm -fv $(APP_NAME)
+	sleep $(TST_W8S1)
+	docker run -d --name $(APP_NAME) $(TST_REPO):$(call _ver,$(TST_VER),full)
+	sleep $(TST_W8L1)
+	docker container logs $(APP_NAME) | grep 'docker-entrypoint.sh'
+	docker rm -fv $(APP_NAME)
+	sleep $(TST_W8S1)
+	#
+	#
+	# test (0) success ☺
+	#
+	#
+	#
+
+test-up_1: test-up-net test-up-deps_1 test-up-auth_1
+	#
+	#
+	#
+	# test (1) ldap auth, sql db, and mail send recv
+	#
+	# send: curl lmtp://app -> srv mysql://db & srv ldap://auth
+	# recv: curl pop3://app
+	# recv: curl imap://app
+	#
+	#
+	docker run -d $(APP_ENV) $(APP_VOL) $(TST_REPO):$(call _ver,$(TST_VER),core)
+
+test-up_2: test-up-net test-up-deps_2 test-up-auth_2
+	#
+	#
+	#
+	# test (2) ldap auth, sql db, mail send recv secure using pem cert files
+	#
+	# send: curl lmtp://app -> srv mysql://db & srv ldap://auth
+	# recv: curl pop3s://app
+	# recv: curl imaps://app
+	#
+	#
+	docker run -d $(APPS_ENV) $(APP_VOL) $(TST_REPO):$(call _ver,$(TST_VER),core)
+
+test-up_3: test-up-net test-up-deps_3 test-up-auth_3 acme/acme.json
+	#
+	#
+	#
+	# test (3) ldap auth, sql db, mail send recv secure using acme cert
+	#
+	# send: curl lmtp://app -> srv mysql://db & srv ldap://auth
+	# recv: curl pop3s://app
+	# recv: curl imaps://app
+	# web: curl http://app
+	#
+	#
+	docker run -d $(APPA_ENV) $(APP_VOL) $(TST_REPO):$(call _ver,$(TST_VER),full)
+
+test-up: test-up_1
+
+test-service: test-service_0
+
+test-service_%: test-pop3_% test-imap_% test-http_%
+	#
+	#
+	# test ($*) success ☺
+	#
+	#
+	#
+
+test-logs_%:
+	-docker ps -a
+	-docker container logs $(AUT_NAME)
+	-docker container logs $(DB_NAME)
+	-docker container logs $(APP_NAME) # | grep 'docker-entrypoint.sh'
+
+test-waits_%:
+	case $* in [1-4]) sleep $(TST_W8S1);; *) sleep $(TST_W8S2);; esac
+
+test-waitl_%:
+	case $* in [1-4]) sleep $(TST_W8L1);; *) sleep $(TST_W8L2);; esac
+
+test-up-net:
+	docker network create $(NET_NAME) 2>/dev/null || true
+
+test-down-net:
+	docker network rm $(NET_NAME) 2>/dev/null || true
+
+test-down: test-down_0 test-down-net acme-destroy
+	
+
+test-down_%:
+	@docker rm -fv $(APP_NAME) $(DB_NAME) $(AUT_NAME) $(TAW_NAME)  2>/dev/null || true
+	@if [ $* -ge 0 ]; then sleep $(TST_W8S1); fi
+
+test-up-deps_%:
+	docker run -d $(DB_ENV) $(DB_VOL) $(DB_IMG) $(DB_CMD)
+	docker run -d $(AUT_ENV) $(AUT_VOL) $(AUT_IMG)
+
+test-up-auth_%:
+	sleep $(TST_W8L1)
+	docker run $(TAW_ENV) $(TST_REPO):$(call _ver,$(TST_VER),core) \
+	zcat /usr/share/doc/kopano/kopano.ldif.gz \
+	| docker exec -i $(AUT_NAME) ldapadd -Q
+	echo "$$LDIF_ADD_DATA" | docker exec -i $(AUT_NAME) ldapadd -Q
+
+test-lmtp: test-lmtp_0
+test-lmtp_%:
+	printf "LHLO mx\nMAIL FROM: <test@example.biz>\nRCPT TO: <$(AD_USR_CN)@$(AD_DOM)>\nDATA\nFrom: A tester <test@example.biz>\nTo: <$(AD_USR_CN)@$(AD_DOM)>\nDate: $$(date)\nSubject: $(MAIL_SUB)$*\n$(MAIL_MSG)$*\n.\nQUIT\n"\
+	| nc -C $(call dkr_cnt_ip,$(APP_NAME)) 2003 # > /dev/null
+	case $* in [1-3]) sleep $(TST_W8S1);; [4-9]) sleep $(TST_W8S2);; esac
+
+test-cfg_%:
+	$(eval cfg_s := $(shell [ $* -ge 2 ] && echo s))
+
+# Note: cannot use prereq also in a proper target since it will change the
+# prereq order, eg. avoid the "3" in test-imaps: test-imap_3
+test-imap: test-imap_0
+test-imaps: test-imap_9
+test-imap_%: test-cfg_% test-lmtp_%
+	docker run $(CURL_ENV) $(CURL_IMG) $(CURL_OPT) imap$(cfg_s)://$(APP_NAME)/inbox \
+	--ssl --anyauth -k -X "fetch 1 all" -u $(AD_USR_CN):$(AD_USR_PW) | grep $(GREP_ENV) $(MAIL_SUB)$*
+
+test-pop3: test-pop3_0
+test-pop3s: test-pop3_9
+test-pop3_%: test-cfg_% test-lmtp_%
+	docker run $(CURL_ENV) $(CURL_IMG) $(CURL_OPT) pop3$(cfg_s)://$(APP_NAME)/1 \
+	--ssl --anyauth -k -u $(AD_USR_CN):$(AD_USR_PW) | grep $(GREP_ENV) $(MAIL_SUB)$*
+
+test-http: test-http_9
+test-http_%: test-cfg_%
+	if [ $* -ge 3 ]; then \
+	docker run $(CURL_ENV) $(CURL_IMG) $(CURL_OPT) http://$(APP_NAME) | grep $(GREP_ENV) $(WEB_TIT) \
+	; fi
+
+test-env:
+	docker exec -it $(APP_NAME) env
+
+test-logs:
+	docker container logs $(APP_NAME)
+
+test-sh:
+	docker exec -it $(APP_NAME) sh -c 'exec $$(getent passwd root | sed "s/.*://g")'
+
+test-diff:
+	docker container diff $(APP_NAME)
+
+test-sv:
+	docker exec -it $(APP_NAME) sh -c 'sv status $$SVDIR/*'
+
+test-debugtools:
+	docker exec -it $(APP_NAME) apt-get update
+	docker exec -it $(APP_NAME) apt-get install --yes \
+	less nano ldap-utils htop net-tools lsof iputils-ping strace
+
+test-htop: test-debugtools
+	docker exec -it $(APP_NAME) htop
+
+test-tls: #--starttls imap
+	docker run --rm -it $(NET_ENV) drwetter/testssl.sh $(APP_NAME):993 || true
+
+acme-destroy: ssl-destroy
+	rm -f acme/*
+
+acme/acme.json: $(APP_CERT)
+	bin/gen-acme-json.sh $(AD_USR_CN)@$(AD_DOM) $(APP_FQDN) $(APP_KEY) $(APP_CERT) > $@
--- a/test/acme/.gitignore
+++ b/test/acme/.gitignore
@ -0,0 +1,4 @@
+# Ignore everything in this directory
+*
+# Except this file
+!.gitignore
--- a/test/ad.mk
+++ b/test/ad.mk
@ -0,0 +1,38 @@
+# ad.mk
+#
+# AD and LDAP   make-functions
+#
+
+#
+# chars
+#
+char_null  :=
+char_space := $(char_null) #
+char_comma := ,
+char_dot   := .
+char_colon := :
+
+#
+# $(call ad_sub_dc,example.com) -> dc=example,dc=com
+#
+ad_sub_dc   = $(subst $(char_space),$(char_comma),$(addprefix dc=, $(subst ., ,$(1))))
+#
+# $(call ad_sub_dot,dc=example,dc=com) -> example.com
+#
+ad_sub_dot  = $(subst $(char_comma)dc=,$(char_dot),$(patsubst dc=%,%,$(1)))
+#
+# $(call ad_cat_dn,admin,dc=example,dc=com) -> cn=admin,dc=example,dc=com
+#
+ad_cat_dn   = cn=$(1),$(2)
+#
+# $(call ad_cut_dot,1,1,example.com) -> example
+#
+ad_cut_dot  = $(subst $(char_space),$(char_dot),$(wordlist $(1), $(2), $(subst $(char_dot),$(char_space),$(3))))
+#
+# $(call ad_rootdc,2,9,adm.dom.org:secret) -> dom.org
+#
+ad_rootdc   = $(subst $(char_space),$(char_dot),$(wordlist $(1), $(2), $(subst $(char_dot),$(char_space),$(firstword $(subst $(char_colon),$(char_space),$(3))))))
+#
+# $(call ad_rootpw,adm.dom.org:secret) -> secret
+#
+ad_rootpw   = $(lastword $(subst $(char_colon),$(char_space),$(1)))
--- a/test/bin/gen-acme-json.sh
+++ b/test/bin/gen-acme-json.sh
@ -0,0 +1,44 @@
+#!/bin/bash
+# args: email hostname keyfile certfile
+mail=$1
+host=$2
+keyfile=$3
+certfile=$4
+
+#
+# The "PrivateKey": attribute needs a PKCS#1 key without tags and line breaks
+# "openssl req -newkey rsa" generates a key stored in PKCS#8 so needs conversion
+#
+#acme_strip_tag() { openssl rsa -in $1 | sed '/^-----/d' | sed ':a;N;$!ba;s/\n//g' ;}
+acme_strip_tag() { sed '/^-----/d' $1 | sed ':a;N;$!ba;s/\n//g' ;}
+
+cat <<-!cat
+{
+  "Account": {
+    "Email": "$mail",
+    "Registration": {
+      "body": {
+        "status": "valid",
+        "contact": [
+          "mailto:$mail"
+        ]
+      },
+      "uri": "https://acme-v02.api.letsencrypt.org/acme/acct/$RANDOM"
+    },
+    "PrivateKey": "$(acme_strip_tag $keyfile)",
+    "KeyType": "2048"
+  },
+  "Certificates": [
+    {
+      "Domain": {
+        "Main": "$host",
+        "SANs": null
+      },
+      "Certificate": "$(base64 -w 0 $certfile)",
+      "Key": "$(base64 -w 0 $keyfile)"
+    }
+  ],
+  "HTTPChallenges": {},
+  "TLSChallenges": {}
+}
+!cat
--- a/test/dkr.mk
+++ b/test/dkr.mk
@ -0,0 +1,34 @@
+# dkr.mk
+#
+# Container   make-functions
+#
+
+#
+# $(call dkr_srv_cnt,app) -> d03dda046e0b90c...
+#
+dkr_srv_cnt = $(shell docker-compose ps -q $(1) | head -n1)
+#
+# $(call dkr_cnt_ip,demo_app_1) -> 172.28.0.3
+#
+dkr_cnt_ip   = $(shell docker inspect -f \
+	'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' \
+	$(1) | head -n1)
+#
+# $(call dkr_srv_ip,app) -> 172.28.0.3
+#
+dkr_srv_ip   = $(shell docker inspect -f \
+	'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' \
+	$$(docker-compose ps -q $(1)) | head -n1)
+#
+#cnt_ip_old = $(shell docker inspect -f \
+#	'{{range .NetworkSettings.Networks}}{{println .IPAddress}}{{end}}' \
+#	$(1) | head -n1)
+
+#
+# List IPs of containers
+#
+ip-list:
+	@for srv in $$(docker ps --format "{{.Names}}"); do \
+	echo $$srv $$(docker inspect -f \
+	'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $$srv); \
+	done | column -t
--- a/test/ssl.mk
+++ b/test/ssl.mk
@ -0,0 +1,125 @@
+# ssl.mk
+#
+# SSL and TLS   make-functions
+#
+
+SSL_O    ?= example.com
+SSL_KEY  ?= rsa:2048 # rsa:2048 rsa:4096
+SSL_MAIL ?=
+SSL_PASS ?= secret
+SSL_SAN  ?=
+SSL_TRST ?=
+
+#
+# Usage: OpenLDAP
+#
+#SSL_O     = $(AD_DOM)
+#target: ssl/auth.crt ssl/demo.crt
+
+#
+# Usage: SMIME
+#
+#SSL_O     = $(MAIL_DOMAIN)
+#SSL_MAIL  = auto
+#SSL_PASS  = $(AD_USR_PW)
+##SSL_TRST  = $(SSL_SMIME)
+#target: ssl/$(AD_USR_CN)@$(MAIL_DOMAIN).p12
+SSL_SMIME =  -setalias "Self Signed SMIME" -addtrust emailProtection \
+	-addreject clientAuth -addreject serverAuth
+
+#
+# Usage: SUbject Alternate Name SAN
+#
+#SSL_O     = example.com
+#SSL_SAN   = "subjectAltName=DNS:auth,DNS:*.docker"
+#target: ssl/auth.crt
+
+
+#
+# $(call ssl_subj,root,example.com,) -> -subj "/CN=root/O=example.com"
+# $(call ssl_subj,root,example.com,auto) -> -subj "/CN=root/O=example.com/emailAddress=root@example.com"
+# $(call ssl_subj,root,example.com,admin@my.org) -> -subj "/CN=root/O=example.com/emailAddress=admin@my.org"
+#
+ssl_subj  = -subj "/CN=$(1)/O=$(2)$(if $(3),/emailAddress=$(if $(findstring @,$(3)),$(3),$(1)@$(2)),)"
+
+#
+# $(call ssl_extfile,"subjectAltName=DNS:auth") -> -extfile <(printf "subjectAltName=DNS:auth")
+#
+ssl_extfile = $(if $(1),-extfile <(printf $(1)),)
+
+
+.PRECIOUS: %.crt %.csr %.key
+SHELL   = /bin/bash
+
+#
+# Personal information exchange file PKCS#12
+#
+%.p12: %.crt
+	openssl pkcs12 -export -in $< -inkey $*.key -out $@ \
+	-passout pass:$(SSL_PASS)
+
+#
+# Certificate PEM
+#
+%.crt: %.csr ssl/ca.crt
+	openssl x509 -req -in $< -CA $(@D)/ca.crt -CAkey $(@D)/ca.key -out $@ \
+	$(call ssl_extfile,$(SSL_SAN)) $(SSL_TRST) -CAcreateserial
+
+#
+# Certificate signing request PEM
+#
+%.csr: ssl
+	openssl req -new -newkey $(SSL_KEY) -nodes -keyout $*.key -out $@ \
+	$(call ssl_subj,$(*F),$(SSL_O),$(SSL_MAIL))
+
+#
+# Certificate authority certificate PEM
+#
+ssl/ca.crt: ssl
+	openssl req -x509 -new -newkey $(SSL_KEY) -nodes -keyout ssl/ca.key -out $@ \
+	$(call ssl_subj,root,$(SSL_O),$(SSL_MAIL))
+
+#
+# SSL directory
+#
+ssl:
+	mkdir -p $@
+
+#
+# Remove all files in SSL directory
+#
+ssl-destroy:
+	rm -f ssl/*
+
+#
+# Inspect all files in SSL directory
+#
+ssl-list:
+	@for file in $$(ls ssl/*); do \
+	case $$file in \
+	*.crt) \
+	printf "\e[33;1m%s\e[0m\n" $$file; \
+	openssl x509 -noout -issuer -subject -ext basicConstraints,keyUsage,extendedKeyUsage,subjectAltName -in $$file;; \
+	*.csr) \
+	printf "\e[33;1m%s\e[0m\n" $$file; \
+	openssl req -noout -subject -in $$file;; \
+	*.key) \
+	printf "\e[33;1m%s\e[0m\n" $$file; \
+	openssl rsa -text -noout -in $$file | head -n 1;; \
+	esac \
+	done
+
+ssl-inspect:
+	@for file in $$(ls ssl/*); do \
+	case $$file in \
+	*.crt) \
+	printf "\e[33;1m%s\e[0m " $$file; \
+	openssl x509 -text -noout -certopt no_sigdump,no_pubkey -in $$file;; \
+	*.csr) \
+	printf "\e[33;1m%s\e[0m " $$file; \
+	openssl req -text -noout -reqopt no_sigdump,no_pubkey,ext_default -in $$file;; \
+	*.key) \
+	printf "\e[33;1m%s\e[0m " $$file; \
+	openssl rsa -text -noout -in $$file | head -n 1;; \
+	esac \
+	done
--- a/test/ssl/.gitignore
+++ b/test/ssl/.gitignore
@ -0,0 +1,4 @@
+# Ignore everything in this directory
+*
+# Except this file
+!.gitignore
Author	SHA1	Message	Date
mlan	fe6566c427	- [docker](Dockerfile) Update Z-Push package URL. - [kopano](src/kopano) Now use ubuntu:20.04 in `kopano-webaddr.sh`.	2021-06-26 12:52:24 +02:00
mlan	313d954381	- [docker](ROADMAP.md) Update roadmap.	2021-05-05 17:05:48 +02:00
mlan	08a3e3ec9b	- [docker](Dockerfile) Now use ubuntu:20.04 as base image. - [docker](Makefile) Don't set BLD_ARG in Makefile. - [kopano](src/kopano/plugin/movetopublicldap.py) Need explicitly to set `allow_no_value=True` due to update in `configparser.3.8.py`. - [demo](demo) Update to `mta-apk_list` target.	2021-04-26 09:24:39 +02:00
mlan	0fb026a934	- [demo](demo) Minor fix.	2021-01-29 11:47:53 +01:00
mlan	bd88becc7c	- [kopano](src/kopano) Bug in kopano-dagent out-of-office now fixed. - [demo](demo) Make demo less dependent on that curl and firefox beeing installed on the host. - [demo](demo) Don't expose any ports on the host, avoiding potential conflicts.	2021-01-28 21:53:29 +01:00
mlan	4579fd70ad	- [docker](Makefile) Set BLD_ARG in Makefile.	2021-01-20 14:28:35 +01:00
mlan	70b6ab3a9f	- [test](test) Export TST_REPO and TST_VER.	2021-01-18 23:03:06 +01:00
mlan	e0fc560dd3	- [test](test) Update to use `mlan/openldap:2`. - [demo](demo) Update to use `mlan/openldap:2`.	2021-01-15 19:25:09 +01:00
mlan	36cd823be8	- [kopano](src/kopano/plugin/movetopublicldap.py) Include Kopano copyright and license.	2021-01-11 20:08:00 +01:00
mlan	6e32101634	- [kopano](src/kopano/plugin/movetopublicldap.py) Add sample config file for the kopano-dagent python plugin movetopublicldap.	2021-01-08 18:09:55 +01:00
mlan	b9c2529745	- [kopano](src/kopano/plugin/movetopublicldap.py) Add LDAP support to the [Move to public](https://documentation.kopano.io/kopanocore_administrator_manual/special_kc_configurations.html#move-to-public ) kopano-dagent python plugin.	2021-01-07 23:34:20 +01:00
mlan	f82c64cd47	- [docker](src/docker/bin/docker-config.sh) Allow the crontab support to work also when the file `/etc/kopano/docker-crontab` is missing.	2020-12-30 10:09:10 +01:00
mlan	39fe4e82a1	- [docker](Dockerfile) Make a copy of the kopano-webapp config files.	2020-12-19 00:46:26 +01:00
mlan	3b44ca9122	- [kopano](src/kopano) `MIGRATE_CONFIG=2` make sure WebApp plugins have configuration files in place.	2020-12-18 23:36:32 +01:00
mlan	c5b37073b6	- [docker](README.md) Updated documentation.	2020-12-18 22:50:27 +01:00
mlan	4ccda2a36c	- [kopano](src/kopano/entry.d/10-kopano-common) Allow module based parameter names. - [docker](src/docker/bin/docker-config.sh) Add provision to set up crontab using envvars.	2020-12-18 11:04:09 +01:00
mlan	8e9a9bcf67	- [docker](Dockerfile) Install kopano-archiver in target `core`.	2020-12-16 17:50:59 +01:00
mlan	7ff783c870	- [docker](Dockerfile) Install the smime webapp plugin providing [S/MIME](https://kopano.com/blog/s-mime-plugin-description/). - [docker](Dockerfile) Install the mdm webapp plugin providing [Mobile Device Management](https://documentation.kopano.io/webapp_mdm_manual/). - [demo](demo) Add [S/MIME](https://kopano.com/blog/s-mime-plugin-description/) cert generation. - [kopano](src/kopano/entry.d/10-kopano-common) Fix bug in `kopano_apply_envvars_php()`.	2020-12-16 17:41:13 +01:00
mlan	b76d723d04	- [demo](demo) Now Also add an LDAP kopano-group.	2020-11-23 15:47:11 +01:00
mlan	56857c5528	- [docker](README.md) Minor changes to documenation.	2020-11-19 21:33:59 +01:00
mlan	611d4ae694	- [kopano](src/kopano) Load Kopano LDAP attributes using `LDAP_PROPMAP=`.	2020-11-19 20:02:46 +01:00
mlan	f13d2d2329	- [kopano](src/kopano) Now, use man pages and sample config files to find valid keys to match envvar.	2020-11-17 21:58:33 +01:00
mlan	9d7ed67165	- [kopano](src/kopano) Now create SSL dir if it is missing.	2020-11-17 09:04:57 +01:00
mlan	0451ace4af	- [docker](README.md) Minor changes to documenation.	2020-11-16 14:33:40 +01:00
mlan	dc275ea073	- [test](test/Makefile) Some sleep time in between db and app speeds up overall test times.	2020-11-16 13:43:50 +01:00
mlan	f9719f8a2b	- [travis-ci](.travis.yml) Fixed test errors on Travis-CI, which was due to long SQL db initialization.	2020-11-16 12:25:36 +01:00
mlan	90b8433d3c	- [travis-ci](.travis.yml) Debug errors on Travis-CI	2020-11-16 11:48:57 +01:00
mlan	737c33dea2	- [travis-ci](.travis.yml) Debug errors on Travis-CI	2020-11-16 11:10:36 +01:00
mlan	c2a2733f7e	- [travis-ci](.travis.yml) Debug errors on Travis-CI	2020-11-16 10:15:11 +01:00
mlan	c92a4c0b68	- [travis-ci](.travis.yml) Debug errors on Travis-CI	2020-11-16 09:45:35 +01:00
mlan	230e6953fd	- [kopano](src/kopano) New, `MIGRATE_CONFIG=all` tries to make configs compatible with new versions.	2020-11-15 22:54:49 +01:00
mlan	362f4e3865	- [docker](src/docker/bin/docker-config.sh) Generalized `dc_cond_chown()`. - [kopano](src/kopano) Updated documentation and bug fix of support for secure IMAPS, POP3S and ICALS.	2020-11-15 20:27:46 +01:00
mlan	4b8bc2b445	- [test](test/Makefile) Add SSL build tests.	2020-11-14 21:37:54 +01:00
mlan	bc80ab04a3	- [travis-ci](.travis.yml) Travis CI now run test-all.	2020-11-13 19:37:30 +01:00
mlan	3a357ee94a	- [test](test/Makefile) Arrange build tests.	2020-11-12 22:45:53 +01:00
mlan	8b252367b9	- [acme](src/acme) Added module providing Let's encrypt TLS certificates using ACME. - [acme](src/acme) Fixed ACME module shell (ash and bash) portability.	2020-11-11 19:08:41 +01:00
mlan	162e9ca256	- [kopano](src/kopano) Adding support for secure IMAPS, POP3S and ICALS. - [kopano](src/kopano) Now generate self-signed certs if needed. - [demo](demo) Enable IMAPS, POP3S, ICALS and SMTPS in demo.	2020-11-10 15:44:38 +01:00