Note: Basically, everything in domainreg is done through domainreg transactions. When you check availability of a domain, new transaction for that command is added to the queue. Then for each tld transactions are picked up from the queue and executed.
When domainreg runs into some problems, usually one of the following two cases happen:
1. Domain search doesn't work for majority (or all) of the TLDs.
2. Domain search doesn't work for minority (one or two) of the TLDS.
How to troubleshoot:
Both cases can be troubleshooted and fixed in the same way. For both cases, domainreg transactions are usually pilled up because rate of processing existing transactions is lower than the rate of creating new ones. For the case 1, processing of transactions is slow usually because registrar is responding slowly or due to network issues or slowness. For case 2, usually process(one or more) that is in charge of those tlds that stopped working, is stuck and stopped processing transactions.
To check number of uncompleted transactions you can execute following command on domainreg machine:
> sudo -u postgres psql domainreg -c "select t.tld_id, tld.tld, count(tld_id) from transaction t inner join tld tld on tld.id = t.tld_id where t.completed is null group by t.tld_id, tld.tld having count(tld_id) > 0;"
Result will show tld id, tld value and number of open transactions for that tld. Number of open transactions per tld may vary but should be close to zero. Everything above 20 per tld is alarming and everything above 50 means that there is definitely something wrong.
For case 1, all tlds should have high number of uncompleted transactions and for case 2, only tlds that don't work will have high number of uncompleted transactions.
How to fix:
1. Domain search doesn't work for majority (or all) of the TLDs.
When registrar is slow in responding or there are network issues, problem will be present until everything is back to normal. However, even after registrar or network is back to normal, domain search will still not work until domainreg processes all pilled up transactions. This can take some time, depending on the number of pilled up transactions, and to speed things you can delete irrelevant transactions (because domain availability check from an hour ago is not really relevant any more). How to delete transactions is described below.
2. Domain search doesn't work for minority (one or two) of the TLDS.
When only few tlds don't work, it usually means that processes for those tlds are stuck. Restarting the domainreg will unstuck the processes and transactions will start to be processed again.
How to restart the domainreg:
> service atomiadomainregistration-api restart
If transactions got pilled up, it can take some time for domainreg to process them and for everything to start working normally. Like in previous case, you can speed things up by deleting transactions for that tld (described below.)
How to delete uncompleted transactions:
To delete transactions, you first need to stop domainreg by executing:
> service atomiadomainregistration-api stop
You can check how many transaction will be deleted before deleted with this command:
> sudo -u postgres psql domainreg -c "select count(*) from transaction where completed is null and command in ('DomainAvailableBulk', 'PeriodicTask', 'PeriodicTaskBulk', 'DomainIsLocal', 'DomainCheck');"
Then you delete the transactions by executing following command. This deletes transactions for all tlds. Use with caution!:
> sudo -u postgres psql domainreg -c "delete from transaction where completed is null and command in ('DomainAvailableBulk', 'PeriodicTask', 'PeriodicTaskBulk', 'DomainIsLocal', 'DomainCheck');"
If you want do delete transactions only for specific tlds (for example com and net), you can do this with adding tld_id condition in previous commad. Replace 6 and 3 with ids of tlds you want to clean (where for example, 6 and 3 are tld ids for com and net, respectively). Use with caution!:
> sudo -u postgres psql domainreg -c "delete from transaction where completed is null and command in ('DomainAvailableBulk', 'PeriodicTask', 'PeriodicTaskBulk', 'DomainIsLocal', 'DomainCheck') and tld_id in (6, 3);"
Start domainreg again after all this:
> service atomiadomainregistration-api start