[ALL DEVICES] Private DNS broken with Let's Encrypt even on new devices

Search This thread

rlees85

Senior Member
Mar 19, 2008
162
77
So today's been a big day, Let's Encrypt original CA expired at around 15:15 UK which is precisely when private DNS on my phone decided it wasn't going to play anymore.


The interesting thing is that Android trusts the new Let's Encrypt CA. Sure enough, browsing to the private DNS server in a web browser works fine. DavDroid/etc also work fine (same certificate used on that endpoint).

It appears that private DNS uses its very own CA certificate trust and this IS affected by the Let's Encrypt change.

So if your private DNS server stopped working with Android at around 15:15 today then check the server is using a Let's Encrypt certificate. This will be your problem.

Anyone know where I should file this bug with AOSP please do let me know. I have searched long and hard and really want this fixed!
 

leo10ui

New member
Sep 12, 2017
1
0
I'm having the same issue, my tls dns server stopped to work on android, any solution?
 

guitphreak

Member
Mar 14, 2006
25
0
Oh dear, I spent the whole day debugging my DoT adguard instance today. Following while hoping there's a solution to this
 

balboah

New member
Oct 1, 2021
1
0
Same issue, right after launching the service. Tricky timing!
I'm wondering how to solve this in a different manner than having to replace the cert with non-letsencrypt
 

guitphreak

Member
Mar 14, 2006
25
0
I'll have a look at alternative certificates for that VM. Google comes up with ZeroSSL. Seams like it'll be tonight's project
 

Tetsumaki

Member
Oct 24, 2015
11
4
Last edited:
  • Like
Reactions: Bipe

Hoerli

Member
Oct 18, 2014
39
19
www.hoerli.net
I have exactly the same problem. I have rented several vServers on which I have installed Pi-Hole and since yesterday ~8:00 (UTC) DNS over TLS no longer works. Have used nginx as proxy. My SSL certificates are all up to date and valid. With the tool Stubby (tested on Windows) I can connect and send DNS queries, but Android does not want.

Unfortunately, this does not seem to work. The parameter
Code:
--preferred-chain="ISRG Root X1"
does not work for certbot.
Code:
certbot: error: unrecognized arguments: --preferred-chain ISRG Root X1

But if I add the parameter
Code:
preferred_chain = ISRG Root X1
in
Code:
/etc/letsencrypt/renewal/domain.conf
, the certificate will be issued via ISRG Root X1.
After creating, the entry disappeared from the configuration.

However, Android 10 and 11 still can't connect.
Firefox says that everything is fine with the certificate.
 

Tetsumaki

Member
Oct 24, 2015
11
4
I have exactly the same problem. I have rented several vServers on which I have installed Pi-Hole and since yesterday ~8:00 (UTC) DNS over TLS no longer works. Have used nginx as proxy. My SSL certificates are all up to date and valid. With the tool Stubby (tested on Windows) I can connect and send DNS queries, but Android does not want.

Unfortunately, this does not seem to work. The parameter
Code:
--preferred-chain="ISRG Root X1"
does not work for certbot.
Code:
certbot: error: unrecognized arguments: --preferred-chain ISRG Root X1

But if I add the parameter
Code:
preferred_chain = ISRG Root X1
in
Code:
/etc/letsencrypt/renewal/domain.conf
, the certificate will be issued via ISRG Root X1.
After creating, the entry disappeared from the configuration.

However, Android 10 and 11 still can't connect.
Firefox says that everything is fine with the certificate.
Which version of certbot client ?
Require 1.6.0 and more.
Better if 1.12.0 and more.
Latest is 1.19.0 now.

Check Certificate chain with: openssl s_client -connect yourdomain.tld:853

For me:


---
Certificate chain
0 s:CN = mydomain.tld
i:C = US, O = Let's Encrypt, CN = R3
1 s:C = US, O = Let's Encrypt, CN = R3
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
---
 
Last edited:
  • Like
Reactions: Hoerli

Hoerli

Member
Oct 18, 2014
39
19
www.hoerli.net
Which version of certbot client ?
Require 1.6.0 and more.
Better if 1.12.0 and more.
Latest is 1.19.0 now.

Check Certificate chain with: openssl s_client -connect yourdomain.tld:853

For me:


---
Certificate chain
0 s:CN = mydomain.tld
i:C = US, O = Let's Encrypt, CN = R3
1 s:C = US, O = Let's Encrypt, CN = R3
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
---
Oh now it seems to work ....

I have only ever used the official package source version.
For Debian 10:
Code:
[email protected]:~# apt-cache policy certbot | grep -i Installed
  Installed: 0.31.0-1+deb10u1

I have now installed the latest version via snap (which I never use).
Code:
name:      certbot
summary:   Automatically configure HTTPS using Let's Encrypt
publisher: Certbot Project (certbot-eff✓)
store-url: https://snapcraft.io/certbot
contact:   https://github.com/certbot/certbot/issues
license:   unset
description: |
  The objective of Certbot, Let's Encrypt, and the ACME (Automated
  Certificate Management Environment) protocol is to make it possible
  to set up an HTTPS server and have it automatically obtain a
  browser-trusted certificate, without any human intervention. This is
  accomplished by running a certificate management agent on the web
  server.

  This agent is used to:
    - Automatically prove to the Let's Encrypt CA that you control the website
    - Obtain a browser-trusted certificate and set it up on your web server
    - Keep track of when your certificate is going to expire, and renew it
    - Help you revoke the certificate if that ever becomes necessary.
commands:
  - certbot
services:
  certbot.renew: oneshot, enabled, inactive
snap-id:      wy7i66qPx4neXr6m9rTh7Y40h8EhtZFh
tracking:     latest/stable
refresh-date: today at 20:31 CEST
channels:
  latest/stable:    1.19.0      2021-09-07 (1434) 44MB classic
  latest/candidate: ↑
  latest/beta:      1.19.0      2021-09-07 (1434) 44MB classic
  latest/edge:      1.20.0.dev0 2021-10-01 (1498) 44MB classic
installed:          1.19.0                 (1434) 44MB classic

The result with the certbot from the package sources:
Code:
Certificate chain
 0 s:CN = mydomain.tld
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3

With snap-version:
Code:
Certificate chain
 0 s:CN = mydomain.tld
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3

Ok, now I ran the command with --preferred-chain="ISRG Root X1" again for all certificates already issued.
Now it works after all

... and .... it works!
The old version of Certbot, has here in 2. place still made entries, which are now gone!
Code:
Certificate chain
 0 s:CN = mydomain.tld
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 
  • Like
Reactions: connectandroid

NightSky256

Senior Member
Mar 6, 2009
401
141
Milano
Thanks! it worked... the only think is that i had to remvoe certbot from apt on my 18.04 and add the one from snap

My pihole is up and running again :D
 
  • Like
Reactions: Hoerli

Top Liked Posts

  • There are no posts matching your filters.
  • 1
    I'm already missing this feature very much. Can we try to install/configure our Android devices to trust ' ISRG Root X1 ' ?

    edit: I just downloaded and trusted x1 on my android (https://letsencrypt.org/certificates/), no succes

    topic on letsencrypt.org: https://community.letsencrypt.org/t/r3-intermediate-certificate-has-expired/160797/108
    1
    Problem solved for me.

    I am using the acme.sh client (3.0.0) and regenerated (delete and recreate) my certificates with this settings :
    --server letsencrypt --preferred-chain "ISRG Root X1"

    More information here : https://github.com/acmesh-official/acme.sh/issues/3723#issuecomment-932143360
    And here: https://github.com/acmesh-official/acme.sh/wiki/Preferred-Chain

    PS: with certbot client:
    certbot renew --preferred-chain "ISRG Root X1" --force-renewal
    1
    I have exactly the same problem. I have rented several vServers on which I have installed Pi-Hole and since yesterday ~8:00 (UTC) DNS over TLS no longer works. Have used nginx as proxy. My SSL certificates are all up to date and valid. With the tool Stubby (tested on Windows) I can connect and send DNS queries, but Android does not want.

    Unfortunately, this does not seem to work. The parameter
    Code:
    --preferred-chain="ISRG Root X1"
    does not work for certbot.
    Code:
    certbot: error: unrecognized arguments: --preferred-chain ISRG Root X1

    But if I add the parameter
    Code:
    preferred_chain = ISRG Root X1
    in
    Code:
    /etc/letsencrypt/renewal/domain.conf
    , the certificate will be issued via ISRG Root X1.
    After creating, the entry disappeared from the configuration.

    However, Android 10 and 11 still can't connect.
    Firefox says that everything is fine with the certificate.
    Which version of certbot client ?
    Require 1.6.0 and more.
    Better if 1.12.0 and more.
    Latest is 1.19.0 now.

    Check Certificate chain with: openssl s_client -connect yourdomain.tld:853

    For me:


    ---
    Certificate chain
    0 s:CN = mydomain.tld
    i:C = US, O = Let's Encrypt, CN = R3
    1 s:C = US, O = Let's Encrypt, CN = R3
    i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
    ---
    1
    Which version of certbot client ?
    Require 1.6.0 and more.
    Better if 1.12.0 and more.
    Latest is 1.19.0 now.

    Check Certificate chain with: openssl s_client -connect yourdomain.tld:853

    For me:


    ---
    Certificate chain
    0 s:CN = mydomain.tld
    i:C = US, O = Let's Encrypt, CN = R3
    1 s:C = US, O = Let's Encrypt, CN = R3
    i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
    ---
    Oh now it seems to work ....

    I have only ever used the official package source version.
    For Debian 10:
    Code:
    [email protected]:~# apt-cache policy certbot | grep -i Installed
      Installed: 0.31.0-1+deb10u1

    I have now installed the latest version via snap (which I never use).
    Code:
    name:      certbot
    summary:   Automatically configure HTTPS using Let's Encrypt
    publisher: Certbot Project (certbot-eff✓)
    store-url: https://snapcraft.io/certbot
    contact:   https://github.com/certbot/certbot/issues
    license:   unset
    description: |
      The objective of Certbot, Let's Encrypt, and the ACME (Automated
      Certificate Management Environment) protocol is to make it possible
      to set up an HTTPS server and have it automatically obtain a
      browser-trusted certificate, without any human intervention. This is
      accomplished by running a certificate management agent on the web
      server.
    
      This agent is used to:
        - Automatically prove to the Let's Encrypt CA that you control the website
        - Obtain a browser-trusted certificate and set it up on your web server
        - Keep track of when your certificate is going to expire, and renew it
        - Help you revoke the certificate if that ever becomes necessary.
    commands:
      - certbot
    services:
      certbot.renew: oneshot, enabled, inactive
    snap-id:      wy7i66qPx4neXr6m9rTh7Y40h8EhtZFh
    tracking:     latest/stable
    refresh-date: today at 20:31 CEST
    channels:
      latest/stable:    1.19.0      2021-09-07 (1434) 44MB classic
      latest/candidate: ↑
      latest/beta:      1.19.0      2021-09-07 (1434) 44MB classic
      latest/edge:      1.20.0.dev0 2021-10-01 (1498) 44MB classic
    installed:          1.19.0                 (1434) 44MB classic

    The result with the certbot from the package sources:
    Code:
    Certificate chain
     0 s:CN = mydomain.tld
       i:C = US, O = Let's Encrypt, CN = R3
     1 s:C = US, O = Let's Encrypt, CN = R3
       i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
     2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
       i:O = Digital Signature Trust Co., CN = DST Root CA X3

    With snap-version:
    Code:
    Certificate chain
     0 s:CN = mydomain.tld
       i:C = US, O = Let's Encrypt, CN = R3
     1 s:C = US, O = Let's Encrypt, CN = R3
       i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
     2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
       i:O = Digital Signature Trust Co., CN = DST Root CA X3

    Ok, now I ran the command with --preferred-chain="ISRG Root X1" again for all certificates already issued.
    Now it works after all

    ... and .... it works!
    The old version of Certbot, has here in 2. place still made entries, which are now gone!
    Code:
    Certificate chain
     0 s:CN = mydomain.tld
       i:C = US, O = Let's Encrypt, CN = R3
     1 s:C = US, O = Let's Encrypt, CN = R3
       i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
    1
    Thanks! it worked... the only think is that i had to remvoe certbot from apt on my 18.04 and add the one from snap

    My pihole is up and running again :D