example.com is just as an example. I do own the domain I am trying to work with.
What am I trying to do?
I want to generate an SSL wildcard certificate using certbot to use on Google Cloud Platform, so that urls like https://this.example.com work.
What have I done?
I have an app engine instance running that uses a custom domain name (example.com). I have a DNS zone in Cloud DNS with example.com as DNS name. A/AAAA records pointing to the same IPv4 & IPv6 addresses as are specified in the custom domain tab of App Engine.
What works?
Navigating to https://example.com or https://www.example.com works. Navigating to http://example.com or http://www.example.com redirects to https.
What doesn't work?
This is the command I ran:
sudo certbot certonly --dns-google --dns-google-credentials *CREDENTIALS* -d *.example.com
With CREDENTIALS being the location of my service account key file.
After a few minutes, I get an error:
Domain: example.com
Type: unauthorized
Detail: No TXT record found at _acme-challenge.example.com
To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address.
What do I think it could be?
I am hoping it just takes a while to update, but I am not sure.
**EDIT: The logs. It went through an automatic secret remover tool before I shared it here. Let me know if it removed something that you need to see!
**EDIT2: Added new logs & IAM role(s) for SA
The Service Account I am using has DNS Administrator as role.
FILE_LOCATION = Location of credentials file
***** = Hidden
2021-11-25 21:18:03,991:DEBUG:certbot.main:certbot version: 0.40.0
2021-11-25 21:18:03,992:DEBUG:certbot.main:Arguments: ['--dns-google', '--dns-google-credentials', 'FILE_LOCATION', '-d', '*.example.com', '--dns-google-propagation-seconds', '120']
2021-11-25 21:18:03,992:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#dns-google,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2021-11-25 21:18:04,003:DEBUG:certbot.log:Root logging level set at 20
2021-11-25 21:18:04,003:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2021-11-25 21:18:04,004:DEBUG:certbot.plugins.selection:Requested authenticator dns-google and installer None
2021-11-25 21:18:04,004:DEBUG:certbot.plugins.selection:Single candidate plugin: * dns-google
Description: Obtain certificates using a DNS TXT record (if you are using Google Cloud DNS for DNS).
Interfaces: IAuthenticator, IPlugin
Entry point: dns-google = certbot_dns_google.dns_google:Authenticator
Initialized: <certbot_dns_google.dns_google.Authenticator object at 0x7ff1c3afc7c0>
Prep: True
2021-11-25 21:18:04,004:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_dns_google.dns_google.Authenticator object at 0x7ff1c3afc7c0> and installer None
2021-11-25 21:18:04,004:INFO:certbot.plugins.selection:Plugins selected: Authenticator dns-google, Installer None
2021-11-25 21:18:04,008:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-v02.api.letsencrypt.org/acme/acct/*****', new_authzr_uri=None, terms_of_service=None), *****, Meta(creation_dt=datetime.datetime(2021, 11, 24, 22, 26, 59, tzinfo=<UTC>), creation_host='*****'))>
2021-11-25 21:18:04,008:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2021-11-25 21:18:04,010:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org:443
2021-11-25 21:18:04,489:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 658
2021-11-25 21:18:04,491:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 25 Nov 2021 20:18:04 GMT
Content-Type: application/json
Content-Length: 658
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
{
"HgGq9-KQs_w": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
"keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
"meta": {
"caaIdentities": [
"letsencrypt.org"
],
"termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
"website": "https://letsencrypt.org"
},
"newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
"newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
"newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
"revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}
2021-11-25 21:18:04,493:INFO:certbot.main:Obtaining a new certificate
2021-11-25 21:18:04,525:DEBUG:certbot.crypto_util:Generating key (2048 bits): /etc/letsencrypt/keys/0036_key-certbot.pem
2021-11-25 21:18:04,529:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/0036_csr-certbot.pem
2021-11-25 21:18:04,530:DEBUG:acme.client:Requesting fresh nonce
2021-11-25 21:18:04,530:DEBUG:acme.client:Sending HEAD request to https://acme-v02.api.letsencrypt.org/acme/new-nonce.
2021-11-25 21:18:04,747:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2021-11-25 21:18:04,749:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 25 Nov 2021 20:18:04 GMT
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: *****
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
2021-11-25 21:18:04,749:DEBUG:acme.client:Storing nonce: *****
2021-11-25 21:18:04,750:DEBUG:acme.client:JWS payload:
b'{\n "identifiers": [\n {\n "type": "dns",\n "value": "*.example.com"\n }\n ]\n}'
2021-11-25 21:18:04,757:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/new-order:
{
"protected": "*****",
"signature": "*****",
"payload": "*****"
}
2021-11-25 21:18:05,475:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 338
2021-11-25 21:18:05,476:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Date: Thu, 25 Nov 2021 20:18:05 GMT
Content-Type: application/json
Content-Length: 338
Connection: keep-alive
Boulder-Requester: *****
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-v02.api.letsencrypt.org/acme/order/*****/*****
Replay-Nonce: *****
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
{
"status": "pending",
"expires": "2021-12-02T20:18:05Z",
"identifiers": [
{
"type": "dns",
"value": "*.example.com"
}
],
"authorizations": [
"https://acme-v02.api.letsencrypt.org/acme/authz-v3/*****"
],
"finalize": "https://acme-v02.api.letsencrypt.org/acme/finalize/*****/*****"
}
2021-11-25 21:18:05,477:DEBUG:acme.client:Storing nonce: *****
2021-11-25 21:18:05,478:DEBUG:acme.client:JWS payload:
b''
2021-11-25 21:18:05,484:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/*****:
{
"protected": "*****",
"signature": "*****",
"payload": ""
}
2021-11-25 21:18:06,101:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/***** HTTP/1.1" 200 386
2021-11-25 21:18:06,103:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 25 Nov 2021 20:18:06 GMT
Content-Type: application/json
Content-Length: 386
Connection: keep-alive
Boulder-Requester: *****
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: *****
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
{
"identifier": {
"type": "dns",
"value": "example.com"
},
"status": "pending",
"expires": "2021-12-02T20:18:05Z",
"challenges": [
{
"type": "dns-01",
"status": "pending",
"url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/*****/*****",
"token": "*****"
}
],
"wildcard": true
}
2021-11-25 21:18:06,103:DEBUG:acme.client:Storing nonce: *****
2021-11-25 21:18:06,104:INFO:certbot.auth_handler:Performing the following challenges:
2021-11-25 21:18:06,105:INFO:certbot.auth_handler:dns-01 challenge for example.com
2021-11-25 21:18:06,106:WARNING:certbot.plugins.dns_common:Unsafe permissions on credentials configuration file: FILE_LOCATION
2021-11-25 21:18:06,110:INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest
2021-11-25 21:18:06,400:INFO:googleapiclient.discovery:URL being requested: GET https://dns.googleapis.com/dns/v1/projects/*****/managedZones?dnsName=example.com.&alt=json
2021-11-25 21:18:06,401:INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
2021-11-25 21:18:06,403:INFO:oauth2client.client:Refreshing access_token
2021-11-25 21:18:06,796:DEBUG:certbot_dns_google.dns_google:Found id of ***** for example.com using name example.com
2021-11-25 21:18:06,800:INFO:googleapiclient.discovery:URL being requested: GET https://dns.googleapis.com/dns/v1/projects/*****/managedZones/*****/rrsets?alt=json
2021-11-25 21:18:06,998:INFO:googleapiclient.discovery:URL being requested: POST https://dns.googleapis.com/dns/v1/projects/*****/managedZones/*****/changes?alt=json
2021-11-25 21:18:07,676:INFO:googleapiclient.discovery:URL being requested: GET https://dns.googleapis.com/dns/v1/projects/*****/managedZones/*****/changes/46?alt=json
2021-11-25 21:18:07,894:INFO:certbot.plugins.dns_common:Waiting 120 seconds for DNS changes to propagate
2021-11-25 21:20:07,995:INFO:certbot.auth_handler:Waiting for verification...
2021-11-25 21:20:07,996:DEBUG:acme.client:JWS payload:
b'{\n "resource": "challenge",\n "type": "dns-01"\n}'
2021-11-25 21:20:08,008:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/*****/*****:
{
"protected": "*****",
"signature": "*****",
"payload": "*****"
}
2021-11-25 21:20:08,782:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/chall-v3/*****/***** HTTP/1.1" 200 185
2021-11-25 21:20:08,784:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 25 Nov 2021 20:20:08 GMT
Content-Type: application/json
Content-Length: 185
Connection: keep-alive
Boulder-Requester: *****
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index", <https://acme-v02.api.letsencrypt.org/acme/authz-v3/*****>;rel="up"
Location: https://acme-v02.api.letsencrypt.org/acme/chall-v3/*****/*****
Replay-Nonce: *****
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
{
"type": "dns-01",
"status": "pending",
"url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/*****/*****",
"token": "*****"
}
2021-11-25 21:20:08,784:DEBUG:acme.client:Storing nonce: *****
2021-11-25 21:20:09,786:DEBUG:acme.client:JWS payload:
b''
2021-11-25 21:20:09,793:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/*****:
{
"protected": "*****",
"signature": "*****",
"payload": ""
}
2021-11-25 21:20:11,326:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/***** HTTP/1.1" 200 668
2021-11-25 21:20:11,327:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 25 Nov 2021 20:20:11 GMT
Content-Type: application/json
Content-Length: 668
Connection: keep-alive
Boulder-Requester: *****
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: *****
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
{
"identifier": {
"type": "dns",
"value": "example.com"
},
"status": "invalid",
"expires": "2021-12-02T20:18:05Z",
"challenges": [
{
"type": "dns-01",
"status": "invalid",
"error": {
"type": "urn:ietf:params:acme:error:dns",
"detail": "DNS problem: NXDOMAIN looking up TXT for _acme-challenge.example.com - check that a DNS record exists for this domain",
"status": 400
},
"url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/*****/*****",
"token": "*****",
"validated": "2021-11-25T20:20:08Z"
}
],
"wildcard": true
}
2021-11-25 21:20:11,328:DEBUG:acme.client:Storing nonce: *****
2021-11-25 21:20:11,329:WARNING:certbot.auth_handler:Challenge failed for domain example.com
2021-11-25 21:20:11,330:INFO:certbot.auth_handler:dns-01 challenge for example.com
2021-11-25 21:20:11,330:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:
Domain: example.com
Type: dns
Detail: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.example.com - check that a DNS record exists for this domain
2021-11-25 21:20:11,331:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 91, in handle_authorizations
self._poll_authorizations(authzrs, max_retries, best_effort)
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 180, in _poll_authorizations
raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2021-11-25 21:20:11,331:DEBUG:certbot.error_handler:Calling registered functions
2021-11-25 21:20:11,331:INFO:certbot.auth_handler:Cleaning up challenges
2021-11-25 21:20:11,334:INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest
2021-11-25 21:20:12,254:INFO:googleapiclient.discovery:URL being requested: GET https://dns.googleapis.com/dns/v1/projects/*****/managedZones?dnsName=example.com.&alt=json
2021-11-25 21:20:12,254:INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
2021-11-25 21:20:12,256:INFO:oauth2client.client:Refreshing access_token
2021-11-25 21:20:12,832:DEBUG:certbot_dns_google.dns_google:Found id of ***** for example.com using name example.com
2021-11-25 21:20:12,836:INFO:googleapiclient.discovery:URL being requested: GET https://dns.googleapis.com/dns/v1/projects/*****/managedZones/*****/rrsets?alt=json
2021-11-25 21:20:13,009:INFO:googleapiclient.discovery:URL being requested: POST https://dns.googleapis.com/dns/v1/projects/*****/managedZones/*****/changes?alt=json
2021-11-25 21:20:13,628:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File "/usr/bin/certbot", line 11, in <module>
load_entry_point('certbot==0.40.0', 'console_scripts', 'certbot')()
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1382, in main
return config.func(config, plugins)
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1265, in certonly
lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
File "/usr/lib/python3/dist-packages/certbot/main.py", line 121, in _get_and_save_cert
lineage = le_client.obtain_and_enroll_certificate(domains, certname)
File "/usr/lib/python3/dist-packages/certbot/client.py", line 417, in obtain_and_enroll_certificate
cert, chain, key, _ = self.obtain_certificate(domains)
File "/usr/lib/python3/dist-packages/certbot/client.py", line 348, in obtain_certificate
orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
File "/usr/lib/python3/dist-packages/certbot/client.py", line 396, in _get_order_and_authorizations
authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 91, in handle_authorizations
self._poll_authorizations(authzrs, max_retries, best_effort)
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 180, in _poll_authorizations
raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
After a long comment thread, we figured out the problem.
The OP is using Google Domains for the Authoritative Name Servers.
Certbot does not have a plugin for Google Domains DNS servers. Google Domains does not publish an API.
The solution is to manually create the required DNS resource records or switch to another supported DNS server such as Google Cloud DNS.