[ALL DEVICES] Private DNS broken with Let's Encrypt even on new devices

rlees85 · Sep 30, 2021

So today's been a big day, Let's Encrypt original CA expired at around 15:15 UK which is precisely when private DNS on my phone decided it wasn't going to play anymore.

DST Root CA X3 Expiration (September 2021) - Let's Encrypt

Update September 30, 2021 As planned, the DST Root CA X3 cross-sign has expired, and we’re now using our own ISRG Root X1 for trust on almost all devices. For more details about the plan, keep reading! We have also updated our Production Chain Changes thread on our community forum - our team and...

letsencrypt.org

The interesting thing is that Android trusts the new Let's Encrypt CA. Sure enough, browsing to the private DNS server in a web browser works fine. DavDroid/etc also work fine (same certificate used on that endpoint).

It appears that private DNS uses its very own CA certificate trust and this IS affected by the Let's Encrypt change.

So if your private DNS server stopped working with Android at around 15:15 today then check the server is using a Let's Encrypt certificate. This will be your problem.

Anyone know where I should file this bug with AOSP please do let me know. I have searched long and hard and really want this fixed!

leo10ui · Oct 1, 2021

I'm having the same issue, my tls dns server stopped to work on android, any solution?

Tetsumaki · Oct 1, 2021

Same problem with stock device Android 9. And /e/ ROM Android 10 on Gigaset GS290...

guitphreak · Oct 1, 2021

Oh dear, I spent the whole day debugging my DoT adguard instance today. Following while hoping there's a solution to this

v411e · Oct 1, 2021

guitphreak said:
Oh dear, I spent the whole day debugging my DoT adguard instance today. Following while hoping there's a solution to this

Also wasted the whole last day. So sad. This was a really great feature.

balboah · Oct 1, 2021

Same issue, right after launching the service. Tricky timing!
I'm wondering how to solve this in a different manner than having to replace the cert with non-letsencrypt

SamJongenelen · Oct 1, 2021

I'm already missing this feature very much. Can we try to install/configure our Android devices to trust ' ISRG Root X1 ' ?

edit: I just downloaded and trusted x1 on my android (https://letsencrypt.org/certificates/), no succes

topic on letsencrypt.org: https://community.letsencrypt.org/t/r3-intermediate-certificate-has-expired/160797/108

guitphreak · Oct 1, 2021

I'll have a look at alternative certificates for that VM. Google comes up with ZeroSSL. Seams like it'll be tonight's project

Tetsumaki · Oct 1, 2021

Problem solved for me.

I am using the acme.sh client (3.0.0) and regenerated (delete and recreate) my certificates with this settings :
--server letsencrypt --preferred-chain "ISRG Root X1"

More information here : https://github.com/acmesh-official/acme.sh/issues/3723#issuecomment-932143360
And here: https://github.com/acmesh-official/acme.sh/wiki/Preferred-Chain

PS: with certbot client:
certbot renew --preferred-chain "ISRG Root X1" --force-renewal

Hoerli · Oct 1, 2021

I have exactly the same problem. I have rented several vServers on which I have installed Pi-Hole and since yesterday ~8:00 (UTC) DNS over TLS no longer works. Have used nginx as proxy. My SSL certificates are all up to date and valid. With the tool Stubby (tested on Windows) I can connect and send DNS queries, but Android does not want.

Unfortunately, this does not seem to work. The parameter

Code:

--preferred-chain="ISRG Root X1"

does not work for certbot.

Code:

certbot: error: unrecognized arguments: --preferred-chain ISRG Root X1

But if I add the parameter

Code:

preferred_chain = ISRG Root X1

in

Code:

/etc/letsencrypt/renewal/domain.conf

, the certificate will be issued via ISRG Root X1.
After creating, the entry disappeared from the configuration.

However, Android 10 and 11 still can't connect.
Firefox says that everything is fine with the certificate.

Tetsumaki · Oct 1, 2021

Hoerli said:
I have exactly the same problem. I have rented several vServers on which I have installed Pi-Hole and since yesterday ~8:00 (UTC) DNS over TLS no longer works. Have used nginx as proxy. My SSL certificates are all up to date and valid. With the tool Stubby (tested on Windows) I can connect and send DNS queries, but Android does not want.

Unfortunately, this does not seem to work. The parameter

Code:

--preferred-chain="ISRG Root X1"

does not work for certbot.

Code:

certbot: error: unrecognized arguments: --preferred-chain ISRG Root X1

But if I add the parameter

Code:

preferred_chain = ISRG Root X1

in

Code:

/etc/letsencrypt/renewal/domain.conf

, the certificate will be issued via ISRG Root X1.
After creating, the entry disappeared from the configuration.

However, Android 10 and 11 still can't connect.
Firefox says that everything is fine with the certificate.

Which version of certbot client ?
Require 1.6.0 and more.
Better if 1.12.0 and more.
Latest is 1.19.0 now.

Check Certificate chain with: openssl s_client -connect yourdomain.tld:853

For me:

---
Certificate chain
0 s:CN = mydomain.tld
i:C = US, O = Let's Encrypt, CN = R3
1 s:C = US, O = Let's Encrypt, CN = R3
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
---

Hoerli · Oct 1, 2021

Tetsumaki said:
Which version of certbot client ?
Require 1.6.0 and more.
Better if 1.12.0 and more.
Latest is 1.19.0 now.

Check Certificate chain with: openssl s_client -connect yourdomain.tld:853

For me:

---
Certificate chain
0 s:CN = mydomain.tld
i:C = US, O = Let's Encrypt, CN = R3
1 s:C = US, O = Let's Encrypt, CN = R3
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
---

Oh now it seems to work ....

I have only ever used the official package source version.
For Debian 10:

Code:

root@localhost:~# apt-cache policy certbot | grep -i Installed
  Installed: 0.31.0-1+deb10u1

I have now installed the latest version via snap (which I never use).

Code:

name:      certbot
summary:   Automatically configure HTTPS using Let's Encrypt
publisher: Certbot Project (certbot-eff✓)
store-url: https://snapcraft.io/certbot
contact:   https://github.com/certbot/certbot/issues
license:   unset
description: |
  The objective of Certbot, Let's Encrypt, and the ACME (Automated
  Certificate Management Environment) protocol is to make it possible
  to set up an HTTPS server and have it automatically obtain a
  browser-trusted certificate, without any human intervention. This is
  accomplished by running a certificate management agent on the web
  server.

  This agent is used to:
    - Automatically prove to the Let's Encrypt CA that you control the website
    - Obtain a browser-trusted certificate and set it up on your web server
    - Keep track of when your certificate is going to expire, and renew it
    - Help you revoke the certificate if that ever becomes necessary.
commands:
  - certbot
services:
  certbot.renew: oneshot, enabled, inactive
snap-id:      wy7i66qPx4neXr6m9rTh7Y40h8EhtZFh
tracking:     latest/stable
refresh-date: today at 20:31 CEST
channels:
  latest/stable:    1.19.0      2021-09-07 (1434) 44MB classic
  latest/candidate: ↑
  latest/beta:      1.19.0      2021-09-07 (1434) 44MB classic
  latest/edge:      1.20.0.dev0 2021-10-01 (1498) 44MB classic
installed:          1.19.0                 (1434) 44MB classic

The result with the certbot from the package sources:

Code:

Certificate chain
 0 s:CN = mydomain.tld
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3

With snap-version:

Code:

Certificate chain
 0 s:CN = mydomain.tld
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3

Ok, now I ran the command with --preferred-chain="ISRG Root X1" again for all certificates already issued.
Now it works after all

... and .... it works!
The old version of Certbot, has here in 2. place still made entries, which are now gone!

Code:

Certificate chain
 0 s:CN = mydomain.tld
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1

NightSky256 · Oct 4, 2021

Thanks! it worked... the only think is that i had to remvoe certbot from apt on my 18.04 and add the one from snap

My pihole is up and running again

Topics

Topics

Search

[ALL DEVICES] Private DNS broken with Let's Encrypt even on new devices

rlees85

Senior Member

DST Root CA X3 Expiration (September 2021) - Let's Encrypt

leo10ui

New member

Tetsumaki

Member

guitphreak

Member

v411e

New member

balboah

New member

SamJongenelen

New member

guitphreak

Member

Tetsumaki

Member

Hoerli

Member

Tetsumaki

Member

Hoerli

Member

NightSky256

Senior Member

Similar threads

Top Liked Posts

New posts