Author Topic: Supplied TLD is incorrect  (Read 13355 times)

0 Members and 1 Guest are viewing this topic.

Offline Lawrence

  • 573
  • 22
  • Community Manager
  • Active Participant
  • *****
    • Lawrence Wright
Re: Supplied TLD is incorrect
« Reply #15 on: June 12, 2013, 04:17:28 AM »
Voted, thanks for posting such a detailed report!
Skype: sociallarry | AIM: larry.aim@aim.com | Forum Rules & Information

These forums are hosted by me with no intentions to ever monetize them. These forums are here solely for the benfit of the HostBill community.

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Re: Supplied TLD is incorrect
« Reply #16 on: June 13, 2013, 05:19:06 AM »
Adding .mu to perpetual "Not Available" list.

The perpetual "Not Available" issue is affecting the following TLDs:
.ac, .af, .cd, .cx, .ec, .gd, .gs, .gy, .hn, .ht, .io, .lc, .mn, .ms, .mu, .ph, .sh, .tc, .tl, .tm, .vg, .za (org.za, nom.za)

The "Supplied TLD is incorrect" problem is affecting the following TLDs:
.ad, .ao, .ar, .bd, .bh, .bo, .bs, .bw, .bz, .cm, .cr, .cu, .cv, .dj, .gf, .gp, .gt, .iq, .jo, .kw, .ke, .kn, .kw, .lb, .lk, .pk, .sl, .sx, .vn
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline tallship

  • 619
  • 19
  • Active Participant
  • *****
  • Manager Network Services
    • Skype - tallship
    • Business Internet and Telephony since 1985.
Re: Supplied TLD is incorrect
« Reply #17 on: June 13, 2013, 05:54:51 AM »
I believe this issue is caused by HostBill not having any WHOIS database info to query, and failing to an error code.

wtf???

That is WRONG!

Color me stupid, and perhaps that's how a lot of the systems work nowadays, but that is a seriously BROKEN way to do things and can cause major issues - you can actually incur some liability if:

  • Someone registers a domain based on the return of WHOIS data
  • They develop infrastructure, or deposit non-refundable monies w/developers for that domain's/brand's development
  • They discover a few days later that you took their money, said their domain was registered, and now it turns out otherwise
The way it is supposed to work, is:

  • The registry database (NOT the WHOIS database) is queried for availability of the string
  • The API returns available or not, and if it's available the prospective registrant has the option to proceed
  • The prospective registrant completes filling out their contact info for the WHOIS and Registry databases
  • The prospective registrant completes any necessary billing info and submits
  • Your client side software authorizes the charge on their cc
  • Your client side software attempts to register the domain and deducts funds from your reseller account if successful
  • You commit the charge to their card using the cc authorization code if the registration was successful
  • if the registration was not successful (someone could have registered the domain a millisecond before them), they are notified and the your client side software does not commit the charge to their card based on the pre auth.
As far as ccTLDs that require manual intervention by the registry, you might consider setting up two different order pages, one for all of the known totally automated TLDs, and one for the ccTLDs whose registries haven't implemented that level of sophistication, and listing those TLDs with the explanation that said registries are manual and therefore the customer must prepay for the *attempt* at registering the domain.

From there it's a matter of your policy - whether you provide refunds, or merely store credit to be used elsewhere, etc.

Would that work for you?

But man! if kbkp is charging for registrations based on the WHOIS database - that's dangerous for you man. Seriously. All checks for automated services should ONLY be against the registry database itself, while on your other order page, it would be perfectly acceptable (IMO) to say something like, "...requires manual registration services which we do on your behalf for your convenience blah blah.... next we check the WHOIS database to see if it *appears* to be available blah blah, then we charge, then we try, then we notify you in within a fortnight or so, then you're happy, blah blah..."

Remember, even in primitive registries, there are reserved words, prohibited strings, etc... and I am 100% certain that kbkp has not included those exclusions in their modules - this can get you into trouble if the only thing being queried is the WHOIS database of that registry.

Just my two cents bro.

Kindest regards,
Bradley D. Thornton - Manager Network Services, NorthTech Computer   TEL: +1.310.388.9469 (US) | +44.203.318.2755 (UK) | +61.390.088.072 (AU) | +41.43.508.05.10 (CH)
Registered Linux User #190795 - "Ask Bill why the string in [MS-DOS] function 9 is terminated by a dollar sign. Ask him, because he can't answer. Only I know that." - Dr. Gary Kildall.

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Re: Supplied TLD is incorrect
« Reply #18 on: June 13, 2013, 06:46:58 AM »
Actually as it's encrypted, I'm not sure what's being checked.  In WHMCS WHOIS is used.  My assumption is that HostBill doesn't have up-to-date information to check, and doesn't have a fail-to available option, therefore, it fails to one of the listed error codes.

It does cause problems in WHMCS, when someone tries to register a domain like com.co (restricted to registry), or sex.ma (prohibited by Moroccan Law).

Regarding the description you provided TallShip, yes that's exactly what I need, however HostBill doesn't currently do it.  I looked into making a domain module before, but the domains functions are part of the HostBill core and cannot be fixed by anyone except KBKP.

If there was a way to bypass the built-in HostBill domain-handling functions and use a 3rd Party app for domains, I would be doing that right now.  As it is, I have to wait for KBKP to fix this.
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline tallship

  • 619
  • 19
  • Active Participant
  • *****
  • Manager Network Services
    • Skype - tallship
    • Business Internet and Telephony since 1985.
Re: Supplied TLD is incorrect
« Reply #19 on: June 13, 2013, 07:13:40 AM »
I remember having to rewrite portions of the original sekeletal OpenSRS perl code so that we instituted the cc preauth via authorize.net, prior to the attempt to run the registration through, then doing the charge itself if it was successful.

At this time, I'm simply using the OpenSRS HostBill module, and to be completely honest, the likelyhood of someone registering the exact same string your user is wanting to register at the moment they are trying to register that string is pretty slim, unless they were just brainstorming with a buddy and now they're both trying to beat each other to the punch to register it - which happens ;)

Still, it's the sane way to do it, and most order inventory control systems either lock the piece of inventory until the item is purchased or the transaction is abandoned, at which time the inventory item is placed back on the shelf, or do the preauth, acquisition, then commit method.
Bradley D. Thornton - Manager Network Services, NorthTech Computer   TEL: +1.310.388.9469 (US) | +44.203.318.2755 (UK) | +61.390.088.072 (AU) | +41.43.508.05.10 (CH)
Registered Linux User #190795 - "Ask Bill why the string in [MS-DOS] function 9 is terminated by a dollar sign. Ask him, because he can't answer. Only I know that." - Dr. Gary Kildall.

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Re: Supplied TLD is incorrect
« Reply #20 on: June 13, 2013, 07:27:10 AM »
Actually, the HostBill domains system is missing another simple feature I'd like to see, which is a "prohibited domains list".  I requested they implement something like this when I bought the software, and was told to just setup a fake account and put all the domains I didn't want people registering in there.  However, that doesn't stop people from ordering them, as HostBill doesn't check if it is an active domain prior to checking with the registry.

I just decided to see if my HostBill site would stated that com.co is available, and it is.  So I guess HostBill is checking WHOIS records.  Wunderbar!  It's always a fun time explaining that a domain is "really not available".

"Oh, just try it again... I'm sure it is available.  You mustn’t be doing it right."
But I really want it!
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Re: Supplied TLD is incorrect
« Reply #21 on: June 13, 2013, 07:02:49 PM »
Hey, Great News!

It looks like KBKP decrypted the whois.class at some point, and we have direct access at:

/includes/extend/whois/whois.php

I'm going to start working on fixing this ASAP.  If anyone else fixes any TLDs please feel free to post it.
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Fix 1: .ao Domains
« Reply #22 on: June 14, 2013, 12:54:36 AM »
Okay, so if anyone else wants to add .co.ao and .it.ao (Angola) domain support add the following:
HEADS-UP: If you don't know what you're doing, don't do it.

In:
/includes/extend/whois/whois.php

Under:
Code: [Select]
$servers = array(
Add:
Code: [Select]
    '.ao' => Array
        (
        'server' => 'fred.nic.ao',
        'available' => 'No entries found'
    ),

Question to everyone: as KBKP decrypted this file is it fare to say I could upload an updated version for everyone to drop into their sites?

ALSO: If you change the file whois.php, you'll either need to change it again after then next Upgrade (if you use Auto), or add it so the list of files to not upgrade.
« Last Edit: June 14, 2013, 09:12:04 AM by thetrusteeco »
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Fix 2: CoCCA Domains (.af, .cx, .gy, .gs, .ht, .nf, .sb, .tl)
« Reply #23 on: June 14, 2013, 03:28:58 AM »
First off I found 2 more for the perpetual "Not Available" list, .nf and .sb, so:

The lists as shipped in HostBill v4.6.4
The perpetual "Not Available" issue is affecting the following TLDs:
.ac, .af, .cd, .cx, .ec, .gd, .gs, .gy, .hn, .ht, .io, .lc, .mn, .ms, .mu, .nf, .ph, .sb, .sh, .tc, .tl, .tm, .vg, .za (org.za, nom.za)

The "Supplied TLD is incorrect" problem is affecting the following TLDs:
.ad, .ao, .ar, .bd, .bh, .bo, .bs, .bw, .bz, .cm, .cr, .cu, .cv, .dj, .gf, .gp, .gt, .iq, .jo, .kw, .ke, .kn, .kw, .lb, .lk, .pk, .sl, .sx, .vn

Okay so now Fix 2:
In:
/includes/extend/whois/whois.php

Under:
Code: [Select]
$servers = array(
Replace the server data for: .af, .cx, .gy, .gs, .ht, .nf, .sb, and .tl with:

Code: [Select]
        'server' => 'whois.coccaregistry.org',
        'available' => 'Available'

Eg:

Code: [Select]
    '.af' => array(
        'server' => 'whois.nic.af',
        'available' => 'Not Registered'
    ),

Becomes:

Code: [Select]
    '.af' => array(
        'server' => 'whois.coccaregistry.org',
        'available' => 'Available'
    ),

Again: If you don't know what you're doing, don't do it.

The lists after Fix 1 & 2
The perpetual "Not Available" issue is affecting the following TLDs:
.ac, .cd, .ec, .gd, .hn, .io, .lc, .mn, .ms, .mu, .ph, .sh, .tc, .tm, .vg, .za (org.za, nom.za)

The "Supplied TLD is incorrect" problem is affecting the following TLDs:
.ad, .ar, .bd, .bh, .bo, .bs, .bw, .bz, .cm, .cr, .cu, .cv, .dj, .gf, .gp, .gt, .iq, .jo, .kw, .ke, .kn, .kw, .lb, .lk, .pk, .sl, .sx, .vn

KBKP: Feel free to update whois.php using the updates I'm putting in this thread; that's the point.
« Last Edit: June 14, 2013, 09:12:45 AM by thetrusteeco »
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Okay, so some more Fixes
Again: If you don't know what you're doing, don't do it.

In:
/includes/extend/whois/whois.php

Under:
Code: [Select]
$servers = array(
Internet Computer Bureau (.ac, .io, .sh, .tm)

.ac
REPLACE
Code: [Select]
    '.ac' => Array
        (
        'server' => 'whois.nic.ac',
        'available' => '- Available'
    ),

WITH
Code: [Select]
    '.ac' => Array
        (
        'server' => 'http://www.nic.ac/cgi-bin/whois?query=',
        'available' => 'This Domain is available for purchase'
    ),

.io
REPLACE
Code: [Select]
    '.io' => Array
        (
        'server' => 'whois.nic.io',
        'available' => '- Available'
    ),

WITH
Code: [Select]
    '.io' => Array
        (
        'server' => 'http://www.nic.io/cgi-bin/whois?query=',
        'available' => 'This Domain is available for purchase'
    ),

.sh
REPLACE
Code: [Select]
    '.sh' => Array
        (
        'server' => 'whois.nic.sh',
        'available' => '- Available'
    ),

WITH
Code: [Select]
    '.sh' => Array
        (
        'server' => 'http://www.nic.sh/cgi-bin/whois?query=',
        'available' => 'This Domain is available for purchase'
    ),

.tm
REPLACE
Code: [Select]
    '.tm' => Array
        (
        'server' => 'whois.nic.tm',
        'available' => '- Available'
    ),

WITH
Code: [Select]
    '.tm' => Array
        (
        'server' => 'http://www.nic.ac/cgi-bin/whois?query=',
        'available' => 'This Domain is available for purchase'
    ),


Affilias (.bz, .lc, .mn)

.bz
ADD
Code: [Select]
    '.bz' => Array
        (
        'server' => 'whois.afilias-grs.info',
        'available' => 'NOT FOUND'
    ),

.lc
REPLACE
Code: [Select]
    '.lc' => Array(
        'server' => 'http://whois.domaintools.com/',
        'available' => 'NOT FOUND'
    ),

WITH
Code: [Select]
    '.lc' => Array(
        'server' => 'whois.afilias-grs.info',
        'available' => 'NOT FOUND'
    ),

.mn
REPLACE
Code: [Select]
    '.mn' => Array(
        'server' => 'http://whois.domaintools.com/',
        'available' => 'This domain name is not registered'
    ),

WITH
Code: [Select]
    '.mn' => Array(
        'server' => 'whois.afilias-grs.info',
        'available' => 'NOT FOUND'
    ),


Others (.ms, .mu, .ph, .org.za)

.ms
REPLACE
Code: [Select]
    '.ms' => Array
        (
        'server' => 'whois.nic.ms',
        'available' => 'Not Registered'
    ),

WITH
Code: [Select]
    '.ms' => Array
        (
        'server' => 'whois.nic.ms',
        'available' => 'Available'
    ),

.mu
REPLACE
Code: [Select]
    '.mu' => array(
        'server' => 'whois.coccaregistry.org',
        'available' => 'Available'
),

WITH
Code: [Select]
    '.mu' => array(
        'server' => 'whois.nic.mu',
        'available' => 'Available'
),

.ph
REPLACE
Code: [Select]
    '.ph' => Array
        (
        'server' => 'http://bayanhosting.dot.ph/WhoIs.asp?Domain=',
        'available' => 'is still available'
    ),

WITH
Code: [Select]
    '.ph' => Array
        (
        'server' => 'http://www2.dot.ph/WhoIs.asp?Domain=',
        'available' => 'is still available'
    ),

.org.za
UNDER
Code: [Select]
    '.za' => Array
        (
        'server' => 'http://co.za/cgi-bin/whois.sh?Domain=',
        'available' => 'No Matches'
    ),

ADD <<== DO NOT REPLACE
Code: [Select]
    '.org.za' => array
(
        'server' => 'http://www.org.za/cgi-bin/rwhois?domain=',
        'available' => 'Domain not found'
),

The lists after Fix 1, 2 & 3
The perpetual "Not Available" issue is affecting the following TLDs:
.cd, .ec, .gd, .hn, .tc, .vg, .za (nom.za)

The "Supplied TLD is incorrect" problem is affecting the following TLDs:
.ad, .ar, .bd, .bh, .bo, .bs, .bw, .cm, .cr, .cu, .cv, .dj, .gf, .gp, .gt, .iq, .jo, .kw, .ke, .kn, .kw, .lb, .lk, .pk, .sl, .sx, .vn

Question to everyone: as KBKP decrypted this file is it fare to say I could upload an updated version for everyone to drop into their sites?
« Last Edit: June 14, 2013, 09:37:19 AM by thetrusteeco »
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline enddo

  • 36
  • 0
  • Active Participant
  • **
    • Skype - yenicedo
Re: Supplied TLD is incorrect
« Reply #25 on: June 14, 2013, 09:41:43 AM »
Hi

Would it be possible to share the file in every fix :)

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Re: Supplied TLD is incorrect
« Reply #26 on: June 14, 2013, 10:09:36 AM »
Okay enddo, here's an updated whois.php

We can't upload .php file, so it's a .txt file, rename it whois.php, and drop it into /includes/extend/whois/

I am using this on my site right now, running in a CloudLinux, LiteSpeed, MariaDB box.  I doubt you could have a problem, but backup your whois.php file first!  And if you don't know what you're doing don't do it!  If confused, please ask!

I think it is okay to share this as HostBill includes this file non-encrypted.  If anyone thinks this is a violation of KBKP's policies please let me know so I can remove it.

Note: it contains Fix 1 -> 3 (19 tlds + 1 sld)
.ac, .af, .ao, .bz, .cx, .gy, .gs, .ht, .io, .lc, .mn, .ms, .mu, .nf, .ph, .sb, .sh, .tl, .tm, .org.za

There are still many non-functional TLDs I did not get to yet.

Note regarding the file: The only changes I've make are those listed in this thread, except for a brief attempt to alphabetize, which was not completed.  I might alphabetize it later.
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline enddo

  • 36
  • 0
  • Active Participant
  • **
    • Skype - yenicedo
Re: Supplied TLD is incorrect
« Reply #27 on: June 14, 2013, 11:57:41 AM »
there are some 3rd party services
like https://www.robowhois.com/

I think hostbill should have nonregistered TLD option in array
For non registered TLD s system can ask those kind of whois services and return the availabty code.

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Re: Supplied TLD is incorrect
« Reply #28 on: June 15, 2013, 02:24:41 AM »
Hello Enddo, I am not sure how many hostbillers would want to pay for their domain availability checks, so I think the whois.php file should only include free whois checks.

Personally, I would like to use a WHOIS caching service for more obscure TLDs that don't have a live registry/WHOIS database.

domaintools.com ($50/m) looks like a decent option.  It is used in the HostBill raw whois.php and so if you have a domaintools account most of the issues this thread it dealing with wouldn't affect you.  However there doesn't seem to be any documentation on the HostBill site about needing a domainstools account to get HostBill to check availability correctly, so I didn't realize that was the issue until after I started pulling the code apart.  I would rather check with the registry's database than a cache if possible, so I'll be keeping the updated file even if I do sign up with domaintools.

robowhois.com seems to only cache the TLDs that do have a live registry/WHOIS database, so I don't see the value in it.

I'm also considering just building something myself, I'll have to see how much work would be involved before signing up with domainstools.

If anyone knows of any whois-cachers that support rare TLDs, like domaintools does, please let us know so we can check them out.
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz

Offline thetrusteeco

  • 257
  • 11
  • Active Participant
  • *****
    • The Trustee Co.
Fix 4: .bd Domains
« Reply #29 on: June 15, 2013, 08:48:33 AM »
If you want to add .bd (Bangladesh) domains:

In:
/includes/extend/whois/whois.php

UNDER:

Code: [Select]
$servers = array(
ADD:

Code: [Select]
    '.bd' => array(
        'server' => 'http://www.whois.com.bd/?query=',
        'available' => "Domain you requested does not exist."
),

« Last Edit: June 16, 2013, 12:28:29 AM by thetrusteeco »
"No man really becomes a fool until he stops asking questions"
Charles Proteus Steinmetz