Migrating from API v1 to v2
Both versions talk to the same account, balance and submission records with the same API key. You can move one call at a time, run both side by side, and there is no deadline. API v1 stays supported.
Why migrate
- Retry-safe submissions. An optional
Idempotency-Keyheader guarantees a retried request is never charged or queued twice (details). - Batch tracking.
GET /batches/{id}gives one summary with counts per state;GET /batches/{id}/submissionslists the rows. - Machine-readable errors. Insufficient credits is
402 INSUFFICIENT_CREDITS, a bad request is422 VALIDATION_ERROR, and so on, instead of a shared400 SUBMISSION_FAILEDwhose message you had to parse. - Rate-limit visibility.
X-RateLimit-*headers on every response andRetry-Afteron 429. - Richer submit response. Accepted, invalid and duplicate counts, the credit cost, your remaining balance, the new submission ids and the batch summary in one reply.
Comparison
| Feature | API v1 | API v2 |
|---|---|---|
| Base URL | /api/v1 | /api/v2 |
| Authentication | Authorization: Bearer | Authorization: Bearer (same key) |
| Balance | GET /balance.php | GET /balance |
| Submit | POST /submit.php | POST /submissions |
| Submission details | GET /submission/{id} | GET /submissions/{id} |
| Submission list | GET /submissions.php?page=&limit=&status= | GET /submissions?page=&per_page=&status= |
| Batch details | Not available | GET /batches/{id} |
| Batch submissions | Not available | GET /batches/{id}/submissions |
| Idempotency | Duplicate-URL window only (1 min) | Optional Idempotency-Key header, plus the same duplicate window |
| Envelope | {success, data} / {success, error{code,message}} | Same, plus optional error.details |
| Timestamps | 2026-10-23 09:15:02 (UTC) | 2026-10-23T09:15:02Z (UTC, ISO 8601) |
| Error statuses | Mostly 400 with SUBMISSION_FAILED | 402 / 404 / 405 / 409 / 422 / 429 / 503 with specific codes |
| Rate limit | 60/min per account, no headers | Same shared limit, X-RateLimit-* and Retry-After headers |
Unknown status filter | Ignored (unfiltered list) | 422 VALIDATION_ERROR |
Non-string entries in urls | Silently dropped | 422 VALIDATION_ERROR naming the entry |
| Request logging | Per request | Per request (same log) |
Unchanged: API keys, account scope (Reseller keys see their Customers' data), URL validation rules, the duplicate window, batch size limits, credit pricing, charging and refund behaviour, and the background processing that sends URLs to providers. Nothing about how URLs are processed differs between versions.
Authentication
No change. Keep the same header and the same key.
Authorization: Bearer YOUR_API_KEYBalance
curl "https://crawlme.net/api/v1/balance.php" \
-H "Authorization: Bearer YOUR_API_KEY"{
"success": true,
"data": {
"balance": 150.0
}
}curl "https://crawlme.net/api/v2/balance" \
-H "Authorization: Bearer YOUR_API_KEY"{
"success": true,
"data": {
"account_id": 1042,
"balance": 150.0
}
}v2 adds account_id. balance is identical.
Submitting URLs
The request body is the same. The response is richer, and the status code for a replayed request is 200 instead of 201.
curl -X POST "https://crawlme.net/api/v1/submit.php" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"urls":["https://example.com/a","https://example.com/b"]}'{
"success": true,
"data": {
"batch_id": 293244,
"submitted_count": 2,
"invalid": [],
"duplicates": []
}
}curl -X POST "https://crawlme.net/api/v2/submissions" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Idempotency-Key: order-7f3a9c" \
-H "Content-Type: application/json" \
-d '{"urls":["https://example.com/a","https://example.com/b"]}'{
"success": true,
"data": {
"batch_id": 293244,
"replayed": false,
"accepted_count": 2,
"invalid_count": 0,
"duplicate_count": 0,
"credit_cost": 2.0,
"remaining_balance": 148.0,
"invalid": [],
"duplicates": [],
"submission_ids": [
8812001,
8812002
],
"batch": {
"id": 293244,
"account_id": 1042,
"status": "PENDING",
"total": 2,
"successful": 0,
"failed": 0,
"pending": 2,
"queued": 2,
"processing": 0,
"credit_cost": 2.0,
"created_at": "2026-10-23T09:15:02Z",
"completed_at": null
}
}
}| v1 field | v2 field |
|---|---|
| batch_id | batch_id (same value) |
| submitted_count | accepted_count (same meaning) |
| invalid | invalid (same), plus invalid_count |
| duplicates | duplicates (same), plus duplicate_count |
| - | credit_cost, remaining_balance, submission_ids, batch, replayed |
Code to change: read accepted_count instead of submitted_count. Everything else you already read is still there under the same name.
Submission status
curl "https://crawlme.net/api/v1/submission/8812001" \
-H "Authorization: Bearer YOUR_API_KEY"{
"success": true,
"data": {
"id": 8812001,
"batch_id": 293244,
"url": "https://example.com/a",
"status": "SUCCESS",
"credits_used": 1.0,
"submitted_at": "2026-10-23 09:15:02",
"updated_at": "2026-10-23 11:02:11",
"completed_at": "2026-10-23 11:02:11"
}
}curl "https://crawlme.net/api/v2/submissions/8812001" \
-H "Authorization: Bearer YOUR_API_KEY"{
"success": true,
"data": {
"id": 8812001,
"batch_id": 293244,
"account_id": 1042,
"url": "https://example.com/a",
"status": "SUCCESS",
"credits_used": 1.0,
"submitted_at": "2026-10-23T09:15:02Z",
"updated_at": "2026-10-23T11:02:11Z",
"completed_at": "2026-10-23T11:02:11Z"
}
}Same fields and status values; v2 adds account_id and formats timestamps as ISO 8601. If you parse dates, switch the format string. In v2 you can also stop polling individual submissions and watch GET /batches/{id} instead.
Listing submissions
Rename limit to per_page. The pagination object is identical. Rows have the v2 submission shape above. An unknown status value is an error in v2 rather than silently ignored.
GET https://crawlme.net/api/v1/submissions.php?page=2&limit=50&status=FAILEDGET https://crawlme.net/api/v2/submissions?page=2&per_page=50&status=FAILEDError handling
In v1 almost every business failure is 400 SUBMISSION_FAILED and you must read the message. In v2 switch on the status code and error.code:
| Situation | v1 | v2 |
|---|---|---|
| Bad JSON | 400 INVALID_JSON | 400 INVALID_JSON |
| Missing / malformed urls | 400 VALIDATION_ERROR | 422 VALIDATION_ERROR (with details.field) |
| Too many URLs | 400 SUBMISSION_FAILED | 422 BATCH_LIMIT_EXCEEDED |
| Too few valid URLs | 400 SUBMISSION_FAILED | 422 VALIDATION_ERROR (details.minimum_batch_size) |
| All invalid or duplicates | 400 SUBMISSION_FAILED | 422 NO_VALID_URLS (details.invalid, details.duplicates) |
| Not enough credits | 400 SUBMISSION_FAILED | 402 INSUFFICIENT_CREDITS |
| No provider available | 400 SUBMISSION_FAILED | 503 PROVIDER_UNAVAILABLE |
| Idempotency-Key reused with another payload | n/a | 409 IDEMPOTENCY_CONFLICT |
| Unknown endpoint | server 404 page | 404 NOT_FOUND (JSON) |
| Wrong method | 405 METHOD_NOT_ALLOWED | 405 METHOD_NOT_ALLOWED (+ Allow header) |
| Rate limited | 429 RATE_LIMITED | 429 RATE_LIMITED (+ Retry-After) |
| Server error | 500 SERVER_ERROR / 400 "Submission failed. Please try again." | 500 SERVER_ERROR / 500 INTERNAL_ERROR |
Retries
In v1, a retried submission is only safe inside the 1-minute duplicate window. In v2, send an Idempotency-Key that is unique per logical request (a UUID, or your own order id) and reuse it on every retry of that request:
<?php
function submit_with_retry(array $urls, string $idempotencyKey): array
{
for ($attempt = 1; $attempt <= 5; $attempt++) {
$ch = curl_init('https://crawlme.net/api/v2/submissions');
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POST => true,
CURLOPT_HTTPHEADER => [
'Authorization: Bearer YOUR_API_KEY',
'Idempotency-Key: ' . $idempotencyKey,
'Content-Type: application/json',
],
CURLOPT_POSTFIELDS => json_encode(['urls' => $urls]),
CURLOPT_TIMEOUT => 30,
]);
$raw = curl_exec($ch);
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
curl_close($ch);
if ($raw === false || $status >= 500) {
sleep(2 ** $attempt); // network error or server error: retry, same key
continue;
}
$body = json_decode($raw, true);
if ($status === 429) {
sleep(5); // Retry-After header carries the exact wait
continue;
}
return $body; // 201 created, 200 replayed, or a 4xx you should not retry
}
throw new RuntimeException('Gave up after 5 attempts');
}import time, uuid, requests
def submit_with_retry(urls, idempotency_key=None):
idempotency_key = idempotency_key or str(uuid.uuid4())
for attempt in range(1, 6):
try:
r = requests.post(
"https://crawlme.net/api/v2/submissions",
headers={"Authorization": "Bearer YOUR_API_KEY", "Idempotency-Key": idempotency_key},
json={"urls": urls},
timeout=30,
)
except requests.RequestException:
time.sleep(2 ** attempt) # network error: retry with the same key
continue
if r.status_code == 429:
time.sleep(int(r.headers.get("Retry-After", "5")))
continue
if r.status_code >= 500:
time.sleep(2 ** attempt)
continue
return r.json() # 201 created, 200 replayed, or a 4xx you should not retry
raise RuntimeError("Gave up after 5 attempts")async function submitWithRetry(urls, idempotencyKey = crypto.randomUUID()) {
for (let attempt = 1; attempt <= 5; attempt++) {
let response;
try {
response = await fetch("https://crawlme.net/api/v2/submissions", {
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Idempotency-Key": idempotencyKey,
"Content-Type": "application/json",
},
body: JSON.stringify({ urls }),
});
} catch (networkError) {
await new Promise(r => setTimeout(r, 2 ** attempt * 1000)); // retry, same key
continue;
}
if (response.status === 429) {
const wait = Number(response.headers.get("Retry-After") || 5);
await new Promise(r => setTimeout(r, wait * 1000));
continue;
}
if (response.status >= 500) {
await new Promise(r => setTimeout(r, 2 ** attempt * 1000));
continue;
}
return response.json(); // 201 created, 200 replayed, or a 4xx you should not retry
}
throw new Error("Gave up after 5 attempts");
}Checklist
- Change the base URL to
https://crawlme.net/api/v2and drop the.phpsuffixes; rename/submit.phpto/submissionsand/submission/{id}to/submissions/{id}. - In the submit response, read
accepted_countinstead ofsubmitted_count. - Rename the list parameter
limittoper_page. - Parse ISO 8601 timestamps (
2026-10-23T09:15:02Z). - Handle errors by status code and
error.code; stop matching on message text. - Add an
Idempotency-Keyto submissions and retry 429 / 5xx / network errors with the same key. - Optionally replace per-submission polling with
GET /batches/{id}. - Keep the same API key. Nothing to regenerate.
Both versions can run at the same time during the switch. Requests to either count against the same 60 requests/minute account limit.