Friday, August 19, 2005

Managing DNS zone files with dnspython

I've been using dnspython lately for transferring some DNS zone files from one name server to another. I found the package extremely useful, but poorly documented, so I decided to write this post as a mini-tutorial on using dnspython.

Running DNS queries

This is one of the things that's clearly spelled out on the Examples page. Here's how to run a DNS query to get the mail servers (MX records) for dnspython.org:

import dns.resolver

answers = dns.resolver.query('dnspython.org', 'MX')
for rdata in answers:
print 'Host', rdata.exchange, 'has preference', rdata.preference
To run other types of queries, for example for IP addresses (A records) or name servers (NS records), replace MX with the desired record type (A, NS, etc.)

Reading a DNS zone from a file

In dnspython, a DNS zone is available as a Zone object. Assume you have the following DNS zone file called db.example.com:

$TTL 36000
example.com. IN SOA ns1.example.com. hostmaster.example.com. (
2005081201 ; serial
28800 ; refresh (8 hours)
1800 ; retry (30 mins)
2592000 ; expire (30 days)
86400 ) ; minimum (1 day)

example.com. 86400 NS ns1.example.com.
example.com. 86400 NS ns2.example.com.
example.com. 86400 MX 10 mail.example.com.
example.com. 86400 MX 20 mail2.example.com.
example.com. 86400 A 192.168.10.10
ns1.example.com. 86400 A 192.168.1.10
ns2.example.com. 86400 A 192.168.1.20
mail.example.com. 86400 A 192.168.2.10
mail2.example.com. 86400 A 192.168.2.20
www2.example.com. 86400 A 192.168.10.20
www.example.com. 86400 CNAME example.com.
ftp.example.com. 86400 CNAME example.com.
webmail.example.com. 86400 CNAME example.com.

To have dnspython read this file into a Zone object, you can use this code:

import dns.zone
from dns.exception import DNSException

domain = "example.com"
print "Getting zone object for domain", domain
zone_file = "db.%s" % domain

try:
zone = dns.zone.from_file(zone_file, domain)
print "Zone origin:", zone.origin
except DNSException, e:
print e.__class__, e
A zone can be viewed as a dictionary mapping names to nodes; dnspython uses by default name representations which are relative to the 'origin' of the zone. In our zone file, 'example.com' is the origin of the zone, and it gets the special name '@'. A name such as www.example.com is exposed by default as 'www'.

A name corresponds to a node, and a node contains a collection of record dataset, or rdatasets. A record dataset contains all the records of a given type. In our example, the '@' node corresponding to the zone origin contains 4 rdatasets, one for each record type that we have: SOA, NS, MX and A. The NS rdataset contains a set of rdatas, which are the individual records of type NS. The rdata class has subclasses for all the possible record types, and each subclass contains information specific to that record type.

Enough talking, here is some code that will hopefully make the previous discussion a bit clearer:

import dns.zone
from dns.exception import DNSException
from dns.rdataclass import *
from dns.rdatatype import *

domain = "example.com"
print "Getting zone object for domain", domain
zone_file = "db.%s" % domain

try:
zone = dns.zone.from_file(zone_file, domain)
print "Zone origin:", zone.origin
for name, node in zone.nodes.items():
rdatasets = node.rdatasets
print "\n**** BEGIN NODE ****"
print "node name:", name
for rdataset in rdatasets:
print "--- BEGIN RDATASET ---"
print "rdataset string representation:", rdataset
print "rdataset rdclass:", rdataset.rdclass
print "rdataset rdtype:", rdataset.rdtype
print "rdataset ttl:", rdataset.ttl
print "rdataset has following rdata:"
for rdata in rdataset:
print "-- BEGIN RDATA --"
print "rdata string representation:", rdata
if rdataset.rdtype == SOA:
print "** SOA-specific rdata **"
print "expire:", rdata.expire
print "minimum:", rdata.minimum
print "mname:", rdata.mname
print "refresh:", rdata.refresh
print "retry:", rdata.retry
print "rname:", rdata.rname
print "serial:", rdata.serial
if rdataset.rdtype == MX:
print "** MX-specific rdata **"
print "exchange:", rdata.exchange
print "preference:", rdata.preference
if rdataset.rdtype == NS:
print "** NS-specific rdata **"
print "target:", rdata.target
if rdataset.rdtype == CNAME:
print "** CNAME-specific rdata **"
print "target:", rdata.target
if rdataset.rdtype == A:
print "** A-specific rdata **"
print "address:", rdata.address
except DNSException, e:
print e.__class__, e

When run against db.example.com, the code above produces this output.

Modifying a DNS zone file

Let's see how to add, delete and change records in our example.com zone file. dnspython offers several different ways to get to a record if you know its name or its type.

Here's how to modify the SOA record and increase its serial number, a very common operation for anybody who maintains DNS zones. I use the iterate_rdatas method of the Zone class, which is handy in this case, since we know that the rdataset actually contains one rdata of type SOA:
   
for (name, ttl, rdata) in zone.iterate_rdatas(SOA):
serial = rdata.serial
new_serial = serial + 1
print "Changing SOA serial from %d to %d" %(serial, new_serial)
rdata.serial = new_serial


Here's how to delete a record by its name. I use the delete_node method of the Zone class:

node_delete = "www2"
print "Deleting node", node_delete
zone.delete_node(node_delete)
Here's how to change attributes of existing records. I use the find_rdataset method of the Zone class, which returns a rdataset containing the records I want to change. In the first section of the following code, I'm changing the IP address of 'mail', and in the second section I'm changing the TTL for all the NS records corresponding to the zone origin '@':

A_change = "mail"
new_IP = "192.168.2.100"
print "Changing A record for", A_change, "to", new_IP
rdataset = zone.find_rdataset(A_change, rdtype=A)
for rdata in rdataset:
rdata.address = new_IP

rdataset = zone.find_rdataset("@", rdtype=NS)
new_ttl = rdataset.ttl / 2
print "Changing TTL for NS records to", new_ttl
rdataset.ttl = new_ttl

Here's how to add records to the zone file. The find_rdataset method can be used in this case too, with the create parameter set to True, in which case it creates a new rdataset if it doesn't already exist. Individual rdata objects are then created by instantiating their corresponding classes with the correct parameters -- such as rdata = dns.rdtypes.IN.A.A(IN, A, address="192.168.10.30").

I show here how to add records of type A, CNAME, NS and MX:
  A_add = "www3"
print "Adding record of type A:", A_add
rdataset = zone.find_rdataset(A_add, rdtype=A, create=True)
rdata = dns.rdtypes.IN.A.A(IN, A, address="192.168.10.30")
rdataset.add(rdata, ttl=86400)

CNAME_add = "www3_alias"
target = dns.name.Name(("www3",))
print "Adding record of type CNAME:", CNAME_add
rdataset = zone.find_rdataset(CNAME_add, rdtype=CNAME, create=True)
rdata = dns.rdtypes.ANY.CNAME.CNAME(IN, CNAME, target)
rdataset.add(rdata, ttl=86400)

A_add = "ns3"
print "Adding record of type A:", A_add
rdataset = zone.find_rdataset(A_add, rdtype=A, create=True)
rdata = dns.rdtypes.IN.A.A(IN, A, address="192.168.1.30")
rdataset.add(rdata, ttl=86400)

NS_add = "@"
target = dns.name.Name(("ns3",))
print "Adding record of type NS:", NS_add
rdataset = zone.find_rdataset(NS_add, rdtype=NS, create=True)
rdata = dns.rdtypes.ANY.NS.NS(IN, NS, target)
rdataset.add(rdata, ttl=86400)

A_add = "mail3"
print "Adding record of type A:", A_add
rdataset = zone.find_rdataset(A_add, rdtype=A, create=True)
rdata = dns.rdtypes.IN.A.A(IN, A, address="192.168.2.30")
rdataset.add(rdata, ttl=86400)

MX_add = "@"
exchange = dns.name.Name(("mail3",))
preference = 30
print "Adding record of type MX:", MX_add
rdataset = zone.find_rdataset(MX_add, rdtype=MX, create=True)
rdata = dns.rdtypes.ANY.MX.MX(IN, MX, preference, exchange)
rdataset.add(rdata, ttl=86400)

Finally, after modifying the zone file via the zone object, it's time to write it back to disk. This is easily accomplished with dnspython via the to_file method. I chose to write the modified zone to a new file, so that I have my original zone available for other tests:

new_zone_file = "new.db.%s" % domain
print "Writing modified zone to file %s" % new_zone_file
zone.to_file(new_zone_file)

The new zone file looks something like this (note that all names have been relativized from the origin):

@ 36000 IN SOA ns1 hostmaster 2005081202 28800 1800 2592000 86400
@ 43200 IN NS ns1
@ 43200 IN NS ns2
@ 43200 IN NS ns3
@ 86400 IN MX 10 mail
@ 86400 IN MX 20 mail2
@ 86400 IN MX 30 mail3
@ 86400 IN A 192.168.10.10
ftp 86400 IN CNAME @
mail 86400 IN A 192.168.2.100
mail2 86400 IN A 192.168.2.20
mail3 86400 IN A 192.168.2.30
ns1 86400 IN A 192.168.1.10
ns2 86400 IN A 192.168.1.20
ns3 86400 IN A 192.168.1.30
webmail 86400 IN CNAME @
www 86400 IN CNAME @
www3 86400 IN A 192.168.10.30
www3_alias 86400 IN CNAME www3

Although it looks much different from the original db.example.com file, this file is also a valid DNS zone -- I tested it by having my DNS server load it.

Obtaining a DNS zone via a zone transfer

This is also easily done in dnspython via the from_xfr function of the zone module. Here's how to do a zone transfer for dnspython.org, trying all the name servers for that domain one by one:

import dns.resolver
import dns.query
import dns.zone
from dns.exception import DNSException
from dns.rdataclass import *
from dns.rdatatype import *

domain = "dnspython.org"
print "Getting NS records for", domain
answers = dns.resolver.query(domain, 'NS')
ns = []
for rdata in answers:
n = str(rdata)
print "Found name server:", n
ns.append(n)

for n in ns:
print "\nTrying a zone transfer for %s from name server %s" % (domain, n)
try:
zone = dns.zone.from_xfr(dns.query.xfr(n, domain))
except DNSException, e:
print e.__class__, e


Once we obtain the zone object, we can then manipulate it in exactly the same way as when we obtained it from a file.

Various ways to iterate through DNS records

Here are some other snippets of code that show how to iterate through records of different types assuming we retrieved a zone object from a file or via a zone transfer:

print "\nALL 'IN' RECORDS EXCEPT 'SOA' and 'TXT':"
for name, node in zone.nodes.items():
rdatasets = node.rdatasets
for rdataset in rdatasets:
if rdataset.rdclass != IN or rdataset.rdtype in [SOA, TXT]:
continue
print name, rdataset

print "\nGET_RDATASET('A'):"
for name, node in zone.nodes.items():
rdataset = node.get_rdataset(rdclass=IN, rdtype=A)
if not rdataset:
continue
for rdataset in rdataset:
print name, rdataset

print "\nITERATE_RDATAS('A'):"
for (name, ttl, rdata) in zone.iterate_rdatas('A'):
print name, ttl, rdata

print "\nITERATE_RDATAS('MX'):"
for (name, ttl, rdata) in zone.iterate_rdatas('MX'):
print name, ttl, rdata

print "\nITERATE_RDATAS('CNAME'):"
for (name, ttl, rdata) in zone.iterate_rdatas('CNAME'):
print name, ttl, rdata
You can find the code referenced in this post in these 2 modules: zonemgmt.py and zone_transfer.py.

11 comments:

farro said...

great power

Anonymous said...

Impressive article. Keep up the great work.

regards
Anand

Anonymous said...

to update a cname:

target = dns.name.from_text("mail.example.com")
rdataset = zone.find_rdataset("mail", rdtype="CNAME")

for rdata in rdataset:
rdata.target = target

to update ns:
rdataset = zone.find_rdataset("@", rdtype="NS")
for rdata in rdataset:
if str(rdata.target) == "ns1.foo.com.": # old ns
rdata.target = dns.name.from_text("ns1.example.com")
elif str(rdata.target) == "ns2.foo.com.": # old ns
rdata.target = dns.name.from_text("ns2.example.com")

johnny halfmoon said...

Thanks for sharing; This was really useful!

JayJay said...

This is great stuff - Thanks! I was wondering if you could post something on how to take a text file with about 100 domains in it, and then print the A records into another file or on the screen? Thanks!

Nick W. said...

Thanks for writing this post, it helped a lot when I was trying to wrap my head around dnspython.

Anonymous said...

can I add a nameserver to resolv.conf file ?

Unknown said...

Great post!

Thanks so much for doing a thorough job of documenting it.

Anonymous said...

Hello, I have a issue with cname add exemple.
In my test at the line:
rdata = dns.rdtypes.ANY.CNAME.CNAME(IN, CNAME, target)

I have this error:
Traceback (most recent call last):
File "./test.py", line 57, in
rdata = dns.rdtypes.ANY.CNAME.CNAME(IN, CNAME, target)
AttributeError: 'module' object has no attribute 'CNAME'

I am in centos6 with:
python-dns-1.11.1-2.el6.noarch
Python 2.6.6

Someone have a idea ?

Unknown said...

This is a great tutorial. I just want to add an example for PTR records:

hostname='foo.example.net.'
target='123'
zone = dns.zone.from_file(f='0.10.10.rev', origin='0.10.10.in-addr.arpa')
rdataset = zone.find_rdataset(target, rdtype=PTR, create=True)
rdata = dns.rdtypes.ANY.PTR.PTR(IN, PTR, dns.name.Name(hostname.split('.')))
rdataset.add(rdata, ttl=86400)

The above will add:
123 86400 IN PTR foo.example.net.
to the zone

Stefan said...

Thanks a lot. It helped me to import DNS data into another script...

Modifying EC2 security groups via AWS Lambda functions

One task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructur...