Selenium with mobile proxies: the setup that doesn't get blocked
Selenium with mobile proxies: the setup that doesn’t get blocked
Your selenium script logs in fine for a day, then every run hits a captcha wall or an account checkpoint. You did not change the code. What changed is that the site noticed every browser session left from one datacenter ip and stopped trusting it. The fix is routing chromedriver through a real mobile ip and making the rest of the browser match. Here is the setup, in order, from someone who drives selenium against a real Singapore farm.
I run that farm. Real phones, real SIMs from Singtel, M1, and StarHub. I point selenium at these proxies for actual account work, so this is where chrome fights you in production, not in theory.
The first wall: chrome ignores proxy auth
Most people lose the first afternoon here. You pass a proxy to chrome and it works, until the proxy needs a username and password. Then it just does not. Chrome ignores proxy credentials passed on the command line, and there is no flag that reliably sticks. Your request leaves from your real ip or it hangs, and selenium logs nothing useful. You end up sure the proxy is broken when chrome simply refused to authenticate.
There are two clean fixes.
A background extension. You build a tiny chrome extension on the fly, a manifest plus a background script, that catches the auth challenge and feeds chrome the username and password. You load it into chrome options at startup. It works in headed mode and it is the common answer online. The catch: it does not load in old headless mode, so headless needs the new headless or the second fix.
A local relay. You run a small local proxy that holds the credentials. Chrome points at localhost with no auth, and the relay forwards everything upstream to your authenticated mobile proxy. Chrome never sees a username or password. This is my preference for headless and for running many browsers, because the messy auth lives in one place you control and every browser points at a plain local port.
Wire it in, then verify before anything else
In your selenium setup you add the proxy server to chrome options as a server argument, host and port. Extension route, you also load the extension. Relay route, you point at localhost and the relay’s port. Start the driver.
Then verify, before you write a single line of real automation. Open one page, an ip echo that returns the ip that connected, and read it.
- See your mobile exit ip? You are tunneling.
- See your own office or server ip? The proxy never took. Usually a typo in the server argument or an extension that failed to load.
Guessing here costs an afternoon chasing a bug that was never in your scraping logic. Print the exit ip on every run so the first question, did it even leave from the proxy, answers itself in one second.
Match the fingerprint to the ip
This is the step people skip and then wonder why they still get blocked. The ip is mobile, so the browser has to look mobile too. A selenium chrome reporting a desktop windows user agent while exiting a Singtel mobile ip is a contradiction no real person produces.
- Set a mobile user agent, a recent chrome on android string.
- Set the timezone and language to Singapore so they agree with the ip.
- Keep one consistent story: ip, user agent, timezone, language all pointing at the same kind of user in the same place.
Kill the automation tells
Selenium leaves fingerprints that have nothing to do with the ip. Plain chromedriver sets a navigator flag that reports webdriver is true, and bot checks read it directly. Turn it off with the automation control flag and by excluding the automation switches in chrome options. There are other tells, window size, missing plugins, a headless giveaway in the renderer, but the webdriver flag is the loud one. A real mobile ip carrying an obvious automation flag still gets caught.
Sticky for logins, rotating for crawls
Pick your session model the way you would for any proxy.
- Logged in? Hold a sticky session, one ip for the whole job. An account whose ip jumps mid session looks stolen and trips a security check.
- Wide shallow crawl? A rotating endpoint is fine and spreads the load across pages you touch once.
Selenium does not care either way. It is the proxy endpoint you point at that decides.
The dns leak nobody checks
By default your machine resolves the hostname locally, then connects out through the proxy. The site sees a mobile ip, but your real resolver, often your isp, saw the lookup. For most browser proxying over http this is acceptable. If you route chrome through a socks layer, use the variant that resolves names on the proxy side. The name lookup should happen where the traffic exits, not at home.
Slow down to a human pace
Speed is a tell. A real person on a phone does not fill a form in eighty milliseconds and submit before the page settles. Add small waits, let pages load, move between actions at a pace a thumb could manage. This matters more on mobile than on datacenter, because the whole point of paying for a mobile ip is to look like a human on a phone. Blasting forty actions a second throws that away.
Running many accounts cleanly
If you scale to many browsers, give each its own sticky session rather than sharing one ip. Two selenium instances hitting the same site from the same mobile ip at the same moment looks exactly like a bot farm.
Also give each account its own chrome profile, a separate user data directory, so cookies and local storage never bleed across accounts. Share a profile and the site can tie two accounts together through shared storage even when the ips differ. Pair each profile with its own stable mobile ip. The same account should come back on the same kind of ip every time, not Singtel today and StarHub tomorrow, because a person does not switch carriers between logins.
The honest tradeoffs
Mobile proxies are slower than datacenter and cost more, because real SIMs and real radios cost more. Selenium is heavy, a full browser per session, so it eats memory and cpu a plain http client never would. If your target does not run javascript checks and does not need a real browser, you may not need selenium at all. Use selenium when the site genuinely needs a real browser, and use mobile when the site genuinely blocks datacenter. Paying for both when you need neither is just waste.
What a clean run looks like
Put it together and the run is boring in the best way. Chrome starts, authenticates through the relay, exits on a real Singapore mobile ip, reports a mobile user agent and a Singapore timezone that match, hides the webdriver flag, holds one sticky session for the account, resolves dns at the exit, and moves at a human pace. A session that never stands out is a session that never gets a captcha.
The proxy under it has to be real for any of this to hold. There is a free trial of Singapore Mobile Proxy, real SG SIMs from my own farm, sticky sessions you can hold for a full job, and dns that routes through the tunnel so you stop leaking your real resolver. Point your selenium setup at it and watch the captcha walls stop showing up.
Get new guides and videos first — join the Telegram channel.