URL Indexing API v1 Reference | CrawlMe Developers

API v2 is now available with improved batch tracking, safer retries, and an enhanced developer experience. Existing API v1 integrations will continue working normally. Upgrading is optional. See the v2 reference or the migration guide.

Overview

Base URL:

Base URL

https://crawlme.net/api/v1

Four endpoints. Except for /submission/{id}, paths include the .php suffix.

Method	Path	Purpose
GET	/balance.php	Your credit balance
POST	/submit.php	Submit URLs under your own account and balance
GET	/submission/{id}	One submission's status (also available as /submission.php?id={id})
GET	/submissions.php	Paginated list of your submissions, optionally filtered by status

Every response is JSON with one of two envelopes:

JSON

{"success": true, "data": { ... }}
{"success": false, "error": {"code": "ERROR_CODE", "message": "Human-readable explanation"}}

Timestamps are UTC in YYYY-MM-DD HH:MM:SS form. Monetary values (balance, credits_used) are JSON numbers with two decimals.

Authentication

Send your API key as a Bearer token with every request. Keys are created on the API Access page of your dashboard and start with uidx_. Regenerating a key revokes the previous one immediately.

HTTP

Authorization: Bearer YOUR_API_KEY

The same key also works with API v2. Revoking it, or an Admin disabling API access for the account, closes both versions at once.

Balance

GET`/api/v1/balance.php`#

cURLPHPPythonJavaScript

cURL

curl -X GET "https://crawlme.net/api/v1/balance.php" \
  -H "Authorization: Bearer YOUR_API_KEY"

PHP

<?php
$ch = curl_init("https://crawlme.net/api/v1/balance.php");
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST  => 'GET',
    CURLOPT_HTTPHEADER     => [
    'Authorization: Bearer YOUR_API_KEY',
    ],
]);
$response = json_decode(curl_exec($ch), true);
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
curl_close($ch);

if ($response['success']) {
    print_r($response['data']);
} else {
    echo $status . ' ' . $response['error']['code'] . ': ' . $response['error']['message'];
}

Python

import requests

response = requests.get(
    "https://crawlme.net/api/v1/balance.php",
    headers={
    "Authorization": "Bearer YOUR_API_KEY",
    },
    timeout=30,
)
body = response.json()

if body["success"]:
    print(body["data"])
else:
    print(response.status_code, body["error"]["code"], body["error"]["message"])

JavaScript

const response = await fetch("https://crawlme.net/api/v1/balance.php", {
  method: "GET",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
  },
});
const body = await response.json();

if (body.success) {
  console.log(body.data);
} else {
  console.error(response.status, body.error.code, body.error.message);
}

JSON

{
    "success": true,
    "data": {
        "balance": 150.0
    }
}

Submit URLs

POST`/api/v1/submit.php`#

Body: a JSON object with a urls array. Minimum 1 URLs, maximum 1,000 per call. Non-string entries and blank strings are ignored.

cURLPHPPythonJavaScript

cURL

curl -X POST "https://crawlme.net/api/v1/submit.php" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://example.com/page-1","https://example.com/page-2","not a url"]}'

PHP

<?php
$ch = curl_init("https://crawlme.net/api/v1/submit.php");
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST  => 'POST',
    CURLOPT_HTTPHEADER     => [
    'Authorization: Bearer YOUR_API_KEY',
    'Content-Type: application/json',
    ],
    CURLOPT_POSTFIELDS     => json_encode(['urls' => ['https://example.com/page-1', 'https://example.com/page-2', 'not a url']]),
]);
$response = json_decode(curl_exec($ch), true);
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
curl_close($ch);

if ($response['success']) {
    print_r($response['data']);
} else {
    echo $status . ' ' . $response['error']['code'] . ': ' . $response['error']['message'];
}

Python

import requests

response = requests.post(
    "https://crawlme.net/api/v1/submit.php",
    headers={
    "Authorization": "Bearer YOUR_API_KEY",
    },
    json={"urls":["https://example.com/page-1","https://example.com/page-2","not a url"]},
    timeout=30,
)
body = response.json()

if body["success"]:
    print(body["data"])
else:
    print(response.status_code, body["error"]["code"], body["error"]["message"])

JavaScript

const response = await fetch("https://crawlme.net/api/v1/submit.php", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({"urls":["https://example.com/page-1","https://example.com/page-2","not a url"]}),
});
const body = await response.json();

if (body.success) {
  console.log(body.data);
} else {
  console.error(response.status, body.error.code, body.error.message);
}

Response 201 Created:

JSON

{
    "success": true,
    "data": {
        "batch_id": 293244,
        "submitted_count": 2,
        "invalid": [
            "not a url"
        ],
        "duplicates": []
    }
}

Field	Type	Meaning
batch_id	integer	The batch that groups the accepted URLs. Use it with v2's /batches/{id} if you want a summary.
submitted_count	integer	How many URLs were accepted, queued and charged.
invalid	string[]	Entries that are not well-formed http(s) URLs, echoed exactly as you sent them.
duplicates	string[]	URLs skipped because they repeat an earlier entry in the same call, or were submitted by your account in the last 1 minutes.

Partial acceptance. Valid, non-duplicate URLs are accepted even when others are invalid or duplicates. You are charged 1 credit per accepted URL, all at once. If your balance cannot cover every accepted URL, the call fails and nothing is charged or queued.

Retries. v1 has no idempotency header. If you repeat an identical request within 1 minutes, every URL is reported as a duplicate and nothing is charged. After that window the same URLs are accepted and charged again. For guaranteed retry safety use v2's Idempotency-Key.

One submission

GET`/api/v1/submission/{id}`#

Returns one submission you may see: your own, or (for Reseller keys) one belonging to one of your Customers. Any other id returns 404, whether or not it exists. /submission.php?id={id} is equivalent.

cURLPHPPythonJavaScript

cURL

curl -X GET "https://crawlme.net/api/v1/submission/8812001" \
  -H "Authorization: Bearer YOUR_API_KEY"

PHP

<?php
$ch = curl_init("https://crawlme.net/api/v1/submission/8812001");
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST  => 'GET',
    CURLOPT_HTTPHEADER     => [
    'Authorization: Bearer YOUR_API_KEY',
    ],
]);
$response = json_decode(curl_exec($ch), true);
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
curl_close($ch);

if ($response['success']) {
    print_r($response['data']);
} else {
    echo $status . ' ' . $response['error']['code'] . ': ' . $response['error']['message'];
}

Python

import requests

response = requests.get(
    "https://crawlme.net/api/v1/submission/8812001",
    headers={
    "Authorization": "Bearer YOUR_API_KEY",
    },
    timeout=30,
)
body = response.json()

if body["success"]:
    print(body["data"])
else:
    print(response.status_code, body["error"]["code"], body["error"]["message"])

JavaScript

const response = await fetch("https://crawlme.net/api/v1/submission/8812001", {
  method: "GET",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
  },
});
const body = await response.json();

if (body.success) {
  console.log(body.data);
} else {
  console.error(response.status, body.error.code, body.error.message);
}

JSON

{
    "success": true,
    "data": {
        "id": 8812001,
        "batch_id": 293244,
        "url": "https://example.com/page-1",
        "status": "PROCESSING",
        "credits_used": 1.0,
        "submitted_at": "2026-10-23 09:15:02",
        "updated_at": "2026-10-23 09:18:40",
        "completed_at": null
    }
}

status	Meaning
PENDING	Accepted and charged, waiting for the background worker to send it to a provider.
PROCESSING	A provider has accepted it and is working on it.
SUCCESS	The provider reported success. This does not confirm that search engines have indexed the page.
FAILED	The provider could not process it, or it could not be sent after several attempts. The credit was refunded.
CANCELLED	Reserved value for a submission cancelled before processing. Refunded like FAILED. Not produced by normal API use today.

List submissions

GET`/api/v1/submissions.php`#

Query parameter	Default	Meaning
page	1	Page number, starting at 1.
limit	25	Rows per page, 1 to 100.
status	(none)	Exact-case filter: PENDING, PROCESSING, SUCCESS, FAILED or CANCELLED. Any other value is ignored and the list is unfiltered.

cURLPHPPythonJavaScript

cURL

curl -X GET "https://crawlme.net/api/v1/submissions.php?page=1&limit=25&status=SUCCESS" \
  -H "Authorization: Bearer YOUR_API_KEY"

PHP

<?php
$ch = curl_init("https://crawlme.net/api/v1/submissions.php?page=1&limit=25&status=SUCCESS");
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST  => 'GET',
    CURLOPT_HTTPHEADER     => [
    'Authorization: Bearer YOUR_API_KEY',
    ],
]);
$response = json_decode(curl_exec($ch), true);
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
curl_close($ch);

if ($response['success']) {
    print_r($response['data']);
} else {
    echo $status . ' ' . $response['error']['code'] . ': ' . $response['error']['message'];
}

Python

import requests

response = requests.get(
    "https://crawlme.net/api/v1/submissions.php?page=1&limit=25&status=SUCCESS",
    headers={
    "Authorization": "Bearer YOUR_API_KEY",
    },
    timeout=30,
)
body = response.json()

if body["success"]:
    print(body["data"])
else:
    print(response.status_code, body["error"]["code"], body["error"]["message"])

JavaScript

const response = await fetch("https://crawlme.net/api/v1/submissions.php?page=1&limit=25&status=SUCCESS", {
  method: "GET",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
  },
});
const body = await response.json();

if (body.success) {
  console.log(body.data);
} else {
  console.error(response.status, body.error.code, body.error.message);
}

JSON

{
    "success": true,
    "data": {
        "submissions": [
            {
                "id": 8812002,
                "batch_id": 293244,
                "url": "https://example.com/page-2",
                "status": "SUCCESS",
                "credits_used": 1.0,
                "submitted_at": "2026-10-23 09:15:02",
                "updated_at": "2026-10-23 11:02:11",
                "completed_at": "2026-10-23 11:02:11"
            }
        ],
        "pagination": {
            "page": 1,
            "per_page": 25,
            "total": 1,
            "total_pages": 1
        }
    }
}

Newest first. Reseller keys see their own submissions and their Customers' submissions; Customer keys see only their own.

Response codes

Every status the v1 endpoints return, in the order a request can meet them. Nothing is charged when a call returns an error.

HTTP	code	message	When
200	-	-	Success. data holds the balance, one submission, or a page of submissions.
201	-	-	POST /submit.php created a batch.
503	MAINTENANCE	The API is temporarily unavailable for maintenance.	The platform is in maintenance mode. Retry later.
405	METHOD_NOT_ALLOWED	Use GET for this endpoint. / Use POST for this endpoint.	Wrong HTTP method.
401	UNAUTHENTICATED	Missing or malformed Authorization header. Expected: Authorization: Bearer <api_key>	No Authorization header, or not in the form Bearer <key>.
401	UNAUTHENTICATED	Invalid or revoked API key.	Unknown, revoked or regenerated key, or the account is disabled.
403	FORBIDDEN	API access is not enabled for this account. Contact an Admin to enable it.	The key is valid but API access is switched off for the account.
429	RATE_LIMITED	Too many requests. Please slow down.	More than 60 requests in 60 seconds for this account (v1 and v2 calls combined). Wait and retry. v1 sends no Retry-After header.
400	INVALID_JSON	Request body must be valid JSON.	The body could not be parsed as a JSON object.
400	VALIDATION_ERROR	Request body must include a "urls" array.	urls is missing or not an array.
400	VALIDATION_ERROR	No URLs provided.	urls contains no non-empty string entries.
400	VALIDATION_ERROR	A valid submission id is required.	The id in /submission/{id} is missing or not a positive number.
404	NOT_FOUND	Submission not found.	The submission does not exist or is outside your scope. Both cases give the same answer on purpose.
400	SUBMISSION_FAILED	Maximum N URLs per batch — you submitted M.	More than 1,000 URLs. Split the request.
400	SUBMISSION_FAILED	No active indexing provider is currently configured. Please try again later.	No provider is available right now. Retry later.
400	SUBMISSION_FAILED	No valid, non-duplicate URLs to submit.	Every URL was invalid or a duplicate.
400	SUBMISSION_FAILED	Please submit at least N URLs (only M valid, non-duplicate URLs found).	Fewer valid URLs than the minimum of 1.
400	SUBMISSION_FAILED	Insufficient credits for this submission.	Balance too low (accounts registered directly with the platform).
400	SUBMISSION_FAILED	Insufficient balance for this submission.	Balance too low (Reseller accounts and accounts managed by a Reseller).
400	SUBMISSION_FAILED	Submission failed. Please contact your site administrator.	Accounts managed by a Reseller only: the Reseller's pooled balance cannot cover the request.
400	SUBMISSION_FAILED	Submission failed. Please try again.	Unexpected server error during submission. Retry, and contact support if it persists.
500	SERVER_ERROR	An internal error occurred.	Unexpected server error elsewhere.

Rate limit

60 requests per minute per account, counted over a sliding 60-second window. The limit belongs to the account, not the key, so regenerating a key does not reset it, and requests to API v1 and API v2 count against the same allowance. Rejected requests are not counted.

Overview

Authentication

Balance

GET/api/v1/balance.php#

Submit URLs

POST/api/v1/submit.php#

One submission

GET/api/v1/submission/{id}#

List submissions

GET/api/v1/submissions.php#

Response codes

Rate limit

GET`/api/v1/balance.php`#

POST`/api/v1/submit.php`#

GET`/api/v1/submission/{id}`#

GET`/api/v1/submissions.php`#