Skip to main content

Webparsers.com

Webparsers Homepage
Login
Menu
Menu
Login

How to Fix Cloudflare Error 1020: Complete Guide with Headers, Proxies, and Stealth Browsers

8 min read

Table of Contents

Set Realistic User-Agent and Headers

The next step involves configuring your request headers properly. While the User-Agent is certainly crucial, it’s not the sole indicator that reveals automated bot activity. Several other essential headers play important roles:

Header Purpose
User-Agent Identifies the browser
Accept Tells the server what content types are supported
Accept-Language Language preferences (should match typical browser settings).
Referer Indicates where the request came from (bots often skip it).
Connection Normally keep-alive in browsers.
Sec-Fetch-Site Part of browser fetch metadata (e.g., none, same-origin, etc.).
Sec-Fetch-Mode Typically navigate, cors, etc.
Sec-Fetch-Dest Indicates the destination type (document, script, etc.).
Sec-Fetch-User Present only in top-level navigation with user action (?1).

To include headers in your request, implement them as follows:

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Language": "en-US,en;q=0.5",
    "Connection": "keep-alive"
}


response = requests.get("https://httpbin.org/headers", headers=headers)

If you’re looking for a list of the latest User Agents and want to learn about them, we’ve got a separate post just for that.

However, if you’ve already modified your headers and concealed your bot activity but continue encountering 1020 errors, the issue might stem from Cloudflare detecting you before any headers are transmitted. This detection occurs at the connection level through TLS fingerprinting. In such cases, consider implementing tls-client.

from tls_client import Session


session = Session(client_identifier="chrome_136")
resp = session.get("https://example.com")
print(resp.status_code, resp.text[:100])

During HTTPS connection establishment, your browser or script transmits encryption settings (cipher suites, extensions). Cloudflare analyzes this information to generate a compact fingerprint known as JA3. When the initial HTTP request immediately follows TLS negotiation, header ordering creates an additional fingerprint called JA4.

Standard libraries like requests utilize default OpenSSL configurations or built-in TLS modules, where cipher suites and extension ordering differ significantly from browser implementations. Advanced libraries such as tls-client (demonstrated above) employ browser-style TLS or modify OpenSSL settings to make requests appear more browser-like regarding JA3 fingerprints.

Add Random Delays Between Requests

Another effective technique for making your script behavior appear more human-like and reducing Cloudflare 1020 blocks involves incorporating small delays between requests. Implementing random intervals works even better:

import time 


delay = random.uniform(1.5, 4.0)
time.sleep(delay)

This approach makes your scraping activity appear less suspicious.

Use a Stealth Headless Browser (Selenium, Puppeteer)

If Cloudflare 1020 errors persist, consider implementing a headless browser solution. Continue using proxies and User-Agents, but execute requests through tools like Selenium:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options


options = Options()
options.add_argument(f'--proxy-server=http://user1:pass1@111.111.111.111:8000')
options.add_argument(f'user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36')


driver = webdriver.Chrome(options=options)


driver.get("https://httpbin.org/ip")


driver.quit()

You can also operate the browser in headless mode for background processing. For NodeJS development, Puppeteer or Playwright provide excellent alternatives.

Hide Headless Browser Signals

Cloudflare demonstrates strong capabilities in detecting headless browsers. Standard WebDrivers produce several telltale signals that facilitate easy detection:

navigator.webdriver = true. This JavaScript property gets automatically configured when pages load through WebDriver. In genuine browsers, it remains undefined or false.

Missing chrome.runtime. Chrome extensions utilize window.chrome.runtime for browser communication. In headless environments (particularly with basic WebDriver), this object may be absent or incomplete, triggering errors during access attempts.

Unusual window.outerWidth/outerHeight values. Headless browsers frequently set these equal to innerWidth/innerHeight, or default to fixed dimensions like 800×600. Real devices typically show larger values that don’t match inner dimensions.

No plugins. Genuine users almost always possess at least one plugin. When navigator.plugins.length equals 0, it raises detection flags.

To circumvent these issues, employ stealth plugins or libraries offering “undetectable” modes. These tools conceal navigator.webdriver, simulate authentic plugins and MIME types, add window.chrome objects resembling real Chrome, and address other headless characteristics.

For Python development, try undetected-chromedriver or SeleniumBase (supporting UC-mode, built on undetected-chromedriver):

from seleniumbase import SB


with SB(uc=True, headless=False) as sb:
    url = "https://httpbin.org/headers"
    sb.uc_open_with_reconnect(url, 3)
    html = sb.get_page_source()

This configuration typically bypasses Cloudflare without triggering 1020 errors. For Node.js development, explore puppeteer-extra-plugin-stealth. It patches JavaScript fingerprints and adjusts TLS settings for better browser mimicry.

Integrate Automated Captcha Solvers

Sometimes stealth techniques prove insufficient. WAF systems may enforce challenges (specifically Cloudflare Turnstile) on every request regardless of fingerprint quality. In these scenarios, dedicated captcha solver services become necessary for maintaining automation.

Services like 2Captcha, CapSolver, or Anti-Captcha offer APIs to bypass these obstacles. You extract the sitekey and page URL from target websites and submit them to the solver’s API. The service returns valid tokens for injection into page DOM or request payloads. This enables session continuation as if humans solved the puzzles. While adding latency to request pipelines, robust captcha solvers often represent the only method for accessing high-security pages without manual intervention.

Use a Web Scraping API

While undetectable browsers prove effective, they consume significant resources. Factor in proxy rotation, CAPTCHA-solving services, and Cloudflare’s multi-layered checking approach, and costs escalate rapidly.

Therefore, in many situations, utilizing scraping APIs or dedicated services provides more practical solutions. These tools target protection systems like Cloudflare specifically, continuously evolving to stay current. Additionally, you avoid managing proxies, headless browsers, or maintaining entire technology stacks.

One example is Webparsers’ web scraping API, which handles complete scraping pipelines, from proxy management to CAPTCHA solving, with 99.9% uptime and rapid response times. You only need a Webparsers API key (obtained after registration). Then configure your parameters and retrieve required content:

import requests
import json


api_key = "YOUR-API-KEY"


url = "https://api.hasdata.com/scrape/web"


payload = json.dumps({
  "url": "https://example.com",
  "proxyType": "datacenter",
  "proxyCountry": "US",
  "screenshot": True,
  "jsRendering": True
})
headers = {
  'Content-Type': 'application/json',
  'x-api-key': api_key
}


response = requests.request("POST", url, headers=headers, data=payload)

Check the documentation for comprehensive option listings. When APIs exist for target sites (like Google SERP, Zillow, etc.), use them for time savings.

Full Code Example: Undetectable Headless Browser, Custom Headers, and Delays

Here’s a comprehensive example combining undetectable headless browser mode, custom headers, proxies, and request delays:

from seleniumbase import SB
import random
import time


proxies = [
    "http://111.111.111.111:8000",
    "http://222.222.222.222:8000"
]


user_agents = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36"
]


proxy = random.choice(proxies)
ua = random.choice(user_agents)


with SB(uc=True, headless=False, proxy=proxy, user_agent=ua) as sb:
    url = "https://httpbin.org/headers"
    sb.uc_open_with_reconnect(url, 3)
    
    time.sleep(random.uniform(2.5, 4.5)) 


    html = sb.get_page_source()

This implementation also selects random proxies and User-Agents. Modify the script according to your requirements.

Cloudflare Bypass Test Results and Metrics

To evaluate the effectiveness of these methods, we conducted tests on three websites featuring different Cloudflare protection levels:

Site A. Basic Cloudflare protection, performs bot and script detection, occasionally presents captchas.

Site B. Active blocking enabled for suspicious requests and repeated connections.

Site C. Strict protection featuring advanced bot detection.

You can establish your own site and configure Cloudflare as desired, or create your own testing list. When visiting Cloudflare-protected sites, examine response headers for the cf-ray parameter, which contains the Cloudflare Ray ID.

To reduce variability, each method underwent testing with 1,000 consecutive requests per site. All tests ran on identical hardware under consistent conditions.

Requests received “successful” classification when returning HTTP status code 200 and responses contained expected page elements. Any other status codes (e.g., 403, 1020) or CAPTCHA-requiring responses were marked as “failures.”

Results from testing various methods, both individually and in combination:

Technique Site A. Success Rate (%) Site B. Success Rate (%) Site C. Success Rate (%) Avg. Response Time (S) Avg. CPU (%) Avg. Mem (%)
No Proxy, No Headless 63 42 41 0.44 13.25 54.4
Datacenter Proxy Only 55 30 43 2.32 15.11 56.1
Residential Proxy + Headers 71 68 64 1.36 11.3 53.6
SeleniumBase + UC mode 91 89 77 5.09 78.16 76.1
API-Based (Webparsers API) 99 99 97 4.38 14.4 56.2

Without running scripts, CPU usage remains around 7%, with memory at approximately 37%.

Generally, these tests provide insights into what performs better against Cloudflare protection. However, real-world scraping scenarios typically involve proxy rotation or replacement upon blocking, unlike these controlled tests.

Success rates also depend heavily on site-specific Cloudflare configuration strictness. Some pages immediately present challenges or blocks (error 1020). Others allow several attempts before implementing stricter measures.

Conclusion: Which Method Should You Choose?

The optimal approach depends on your project scale and available team resources. For smaller projects or learning web security intricacies, building and maintaining stealthy browser solutions provides invaluable educational experiences.

However, for large-scale, mission-critical data extraction where reliability and speed are essential, the resource overhead and constant maintenance requirements of DIY solutions often make dedicated web scraping APIs more efficient and cost-effective choices.

Cloudflare’s continuous evolution means this challenge never reaches a permanent solution. You either commit to ongoing adaptation or delegate the responsibility to specialized teams.