One thing that makes Cloudflare worse for home usage is it acts as a termination point for TLS, whereas Tailscale does not. If you use a Tailscale Funnel, you get the TLS certificate on your endpoint. With Cloudflare, they get a TLS certificate for you, and then strip and optionally re-add TLS as traffic passes through them.
I actually have no idea how private networks with WARP are here, but that's a pretty big privacy downgrade for tunneling from the Internet.
I also consider P2P with relay fallback to be highly desirable over always relaying traffic through a third party, too. Firstly, less middlemen. Secondly, it continues working even if the coordination service is unavailable.
I ended up building something in this space recently (TunnelBuddy – https://www.tunnelbuddy.net I’m the author) that lets you use a friend’s machine as an exit node over WebRTC.
One of the design decisions I made was P2P or nothing: there’s a small signalling service, but no TURN/relay servers. If the peers can’t establish a direct connection, the tunnel just doesn’t come up.
The trade-off is fewer successful connections in weird NAT setups, but in return you know your traffic never transits a third-party relay – it goes straight from your client to your friend’s endpoint.
My traffic will transit third parties all the time, since it's going over the Internet. What's the problem with relays, if the traffic is end-to-end encrypted?
- With a TURN/relay, you’re introducing a single, purpose-built box that:
- sees all the tunnel metadata for many users (IP pairs, timing, volume),
- is easy to log at or subpoena/compel,
- and becomes a natural central chokepoint if someone wants to block the system.
- Without that relay, your traffic still crosses random ISPs/routers, but:
- those hops are *generic Internet infrastructure*, not “the TunnelBuddy relay”,
- there’s no extra entity whose whole job is to see everyone’s flows.
I generally prefer tailscale and trust them more than cloudflare to not rug-pull me on pricing, but the two features that push me towards cloudflared is the custom domains and client-less access. I could probably set it up with caddy and some plugins, but then I still need to expose the service and port forward.
I'm definitely not trying to dissuade anyone from using Cloudflare, just making sure people realize the potential privacy implications of doing so. It isn't always obvious, even though some of the features pretty much require it (at least to be handled entirely on Cloudflare's side. You could implement similar features that are split between the endpoint and the coordination server without requiring full TLS stripping. Maybe Tailscale will support some of those as features of the `serve` server?)
> client-less access
JFYI, Tailscale Funnels also work for this, though depending on your use case it may not be ideal. Ultimately, Cloudflare does handle this use case a bit better.
Yeah, because the auth can't be done on Tailscale's end if they don't terminate the TLS connection. However, it is still possible to use an authentication proxy in this situation. Many homelab and small to medium size company setups use OAuth2 Proxy, often with Dex. If you wanted to get fancier, you could use Tailscale for identity when behind the firewall and OAuth2 Proxy when outside the firewall.
This may seem like a lot of effort and it is definitely not nothing, but Cloudflare Tunnels also has a decent number of moving parts and frankly their authentication gateway leaves a bit to be desired for home users.
That's a fair personal decision, but if I would have to put money on it I'd say the chances of new company that raised 160 million of VC funding this year alone vs. established profitable company with a track record of offering free services for many years already I'd put my money on the latter.
> I could probably set it up with caddy and some plugins, but then I still need to expose the service and port forward.
Not so! I have custom domains on my Tailnet with Caddy. You just need to set up Caddy to perform the ACME DNS challenge to get certs for your domain (ironically I use Cloudflare for DNS because they make it very easy to set this up with just an API key). No plugins or open ports needed.
2. Performance and reliability. 100+ PoPs in all major clouds running their data plane routers if they host (still E2EE), or run routers anywhere if you self-host. Dynamic routing to find best paths across the routers.
I don't see any indication that NetFoundry zrok supports end-to-end encryption from the client to the web server. The default configuration definitely terminates SSL on NetFoundry's server, and I don't see any documentation for how to avoid that. There's a TCP tunneling mode, but servers that use this mode can only be accessed by clients that are themselves also connected to the NetFoundry VPN service, not by clients on the public web. What's needed is a TLS tunneling mode that figures out the correct target via SNI, and zrok doesn't seem to provide that.
You are correct, zrok doesn't support mutual TLS. zrok is the free offering that NetFoundry supports so it's easy to see why you looked there for information.
> It'll still terminate TLS at the servers, though. It's not mTLS all the way
> through to the endpoint.
That was the entire point, though. If NetFoundry Frontdoor can see the traffic (because it gets terminated on their servers, mTLS or not), then it's not end-to-end encrypted as the parent commenter claimed.
I think the issue is zrok vs. NetFoundry/OpenZiti. Zrok is the easy button to project a public endpoint from inside a network. It is not encrypted all the way through, as it is a proxy solution. NetFoundry/OpenZiti provides methods to provide tunnels all the way through. NetFoundry is a company, OpenZiti is a FOSS project/technology sponsored by NetFoundry, and zrok is a product of NetFoundry built on OpenZiti tech, so it is easy to cross things up. I think the comment was in regard to NetFoundry/OpenZiti, while your response referenced zrok. The list above has both.
Tunneling p2p with relay fallback is essentially what connet [1] aspires to be. There are a lot of privacy/security benefits exposing endpoints only to participating peers. You can either run it yourself or use hosted version [2].
For tunnels many of the features basically have to work this way, so I'd be surprised if you could avoid it. It's also impossible to avoid if you use normal Cloudflare "protected" DNS entries. You can use Cloudflare as just a DNS server but it's not the default, by default it will proxy everything through Cloudflare, since that's kind of the point. You can't cache HTTP requests you can't see.
Is it technically possible to have something like Tailscale funnel but with something like Cloudflare Access authentication (at least for some options)?
What's the actual win here? Avoiding relay latency in the rare cases Tailscale can't punch through NAT? If that's it, a $3 VPS running Headscale seems simpler. The complexity feels like you're optimizing for the 5% case while adding permanent vendor lock in. What am I missing?
Exactly, just today I set up a cloudflare tunnel to a docker compose service running on my home server. I didn’t want to expose the server directly to the internet, and I want to share this service on a certain domain with broader family.
I have a server at home that works well. I don’t reaaaally want to pay an extra $30-$40/yr and have an extra thing to manage when the CF tunnel works fine for free. I like Tailscale more, but I want to share this with family who won’t install TS and also want to use a specific domain.
What worked for me was handing them a credit card and transitioning myself out of the free tier. (I'd use the free credits they offer prior to doing this - they give you something like $300 immediately on signup.)
The always-free infra remains free, you just have the chance of incurring a bill if you make selections that aren't free or exceed block storage/egress (200GB/10TB) limits of the always-free tier. Leaving the free/trial tier gives you access to a much larger pool of instances. I never successfully deployed an A1 instance prior to becoming a "paying" customer - now I've done it hundreds of times without ever having an issue.
I've been running a small k0s cluster and a standalone webserver for months while incurring about $2.50 - $3 in spending each month, primarily from being slow to remove instance snapshots sitting in block storage.
Even things that are oddly expensive on AWS - like NAT - are free on Oracle. There are zero gotchas.
I hit the same roadblock as the above user and it never occurred to me to just cross the barrier with cash and then scale back to free. Thanks for this.
It doesn't actually charge you anything. You just have to put a card down to be considered a priority because now you potentially can spend money & therefore are more important then the other free-tier losers. /s It's still free tier & still free.
The free tier is also based on capcity usage, and not instances. If you want 3 cores on 1 machine & 1 on another, they're cool with that. I personally run Pangolin on a 1 core & self-hosted github runners on a 3 core.
$3 VPS running Headscale is not simpler since you won't be able to run both headscale and tailscale on your end user machines, I don't recommend it.
The solution we've found is running a white IP container (or VPS) which looks like regular Wireguard outside, while inside it "forwards" to your existing tailscale network.
I don't think you are missing anything. They have a bunch of half baked features like this that aren't as robust as real security vendors and lock you in just like you said.
No. Not all vendors are equal. We can treat ProtonMail differently then Gmail, for example. Looking at what's gone down with VMware, definitely don't get in bed with Broadcom.
> after frustration with Tailscale in environments where it couldn’t properly penetrate NAT/firewall and get a p2p connection, I decided to invest some time into learning something new: Cloudflare Zero Trust + Warp
...which doesn't even try to get a p2p connection. Instead you always get the thing you didn't want. If you're okay with that you could've just ignored how Tailscale connected those devices, that's kind of the point. You've also in the process converted your entire security model to Cloudflare's idea of "Zero Trust" which involves 100% trusting Cloudflare.
The rest of the blog post is fine, but the motivation is honestly baffling.
The difference is that Cloudflare has their own high quality network and PoPs everywhere, so the quality is generally even better than P2P.
This is my experience, we are a fully remote world-wide company and we recently migrated away from Tailscale to Cloudflare and it has been much better.
Peering in Europe is such a mess that even Cloudflare can be pretty bad. Sometimes you have to manually calculate "okay, there's a colo in this particular city that will force the correct route if we proxy all our traffic through it ..."
Is the connection through Cloudflare still encrypted between the two peers, as it would be going through a Tailscale relay? If not, that's definitely a downgrade using the Cloudflare approach. But if not, I'm not sure the trust model is significantly different with maybe the added benefit of the fact that Cloudflare's relay performance is likely better given that relaying traffic is kind of their main thing rather than a very secondary function like it is for Tailscale.
On the other hand, my experience with Tailscale is that they're very, very good at NAT hole punching and I'd rather have a direct connection where possible from a latency standpoint.
I've experimented with Cloudflare tunnels before to sit in front of my Immich instance in my homelab. Only issue is the 100MB upload size for videos. But Immich added upload chunking support to their roadmap so its possible this will work very well in the future.
Immich also has the ability to use different domains for different networks. Meaning that I connect directly to my server when I'm connected to my local home network and connect through cloudflare when I'm out of my house.
With that it becomes clear that some service is self hosted (the DNS record points to a private IP).
It can be a security issue when the Whois record or the domain name allows the identification of the hosting entity. Finding its physical address can be an easy task depending on its social presence.
Then probably the hosting place is an easier target than a data center.
Nice article. For easily exposing private services to the internet I’ve been using https://tuns.sh which lets you run ssh tunnels. It’s nice for a zero install solution.
> it’s against the terms of service to use it for video
It for any large files. They want to limit bandwidth usage but not blanket limit everyone. One user using Plex of Jellyfin probably doesn't move the needle.
Yes and that's what people are criticizing - it's just an arbitrary and thus very bad rule. Completely unrelated to CloudFlare, I streamed a single TV series from a friend's plex account 1-2y ago, that's less traffic than some of my friends use in 2 weeks.
I'm not saying they can't have that rule, it's their infra - I'm just saying that "a boatload of bandwidth" can be anything, depending on who you ask.
Why would anyone expect anyone else to serve video for them for free? Is there any CDN that serves video for you for free???
I'm not even talking about the copyright implications here, just the bandwidth costs. A single movie download would cost more than many hundreds of typical simple HTTP website sessions.
> Why would anyone expect anyone else to serve video for them for free?
I would expect that a freemium service selling encrypted "zero trust" networking should have no idea what traffic is being pushed through my network making enforcement impossible.
Nobody's asking for a free lunch, but the reasonable thing to do would be to simply bandwidth limit freemium accounts across the board, not make exceptions for certain kinds of traffic in what should be a secure network.
Cloudflare does say "video and other large files" so in the end it is about volume, not data type. They probably just want to have the arbitrary decision on specific cases without defining a uniform blanket limit.
Not sure if you are a representative from Cloudflare. But regardless, Cloudflare intentionally made it ambiguous so that they can selectively enforce it.
For sure. But OPs point still stands, people run giant Plex servers and charge for access to the pirated content on them. Huge numbers of users. I imagine they want to enforce on those folks and not people who just happen to have their own running.
You can use Cloudflare for DNS, but you just can’t use them for proxying the traffic. I use Cloudflare with proxying on for everything other than a plex subdomain.
Thanks a lot. Both the post itself and the comments are very useful. I'd like to comment on the content of this post, but it'll take much time for me to wrap my head around.
Tailscale now has the awesome feature of peer relays and now there's no more excuses why you can't traverse that NAT and you can forget about all those DERP servers.
Oh man, someone has to talk about this!! Cory told me about CF's gold issues and it really does seem problematic to me, I'm glad ZT is finally being criticised.
Yes. The acronym is “ZTNA” (Zero Trust Network Access).
It is an alternative to a traditional corporate VPN that addresses a few architectural issues; namely:
- L3 connectivity (which allows for lateral movement) to the corporate network.
- Inbound exposure to the VPN gateway (scaling can become a challenge, not to mention continuous vulnerabilities from… certain vendors)
- Policy management can get convoluted if you want to do micro-segmentation properly.
ZTNA is essentially an “inside-out” architecture and acts (kind of) like a L4 proxy. I’m going to butcher this explanation, but:
1. Company installs apps/VMs/containers throughout their network. These must have network reachability to the internal apps/services the company wants to make available to its users.
2. These apps/VMs/containers establish TLS tunnels back to the company’s tenant in the vendor’s cloud.
3. Company rolls out the vendor’s ZTNA client to user devices. This also establishes a TLS tunnel to the vendor’s cloud. Hence the vendor’s cloud is like a MitM gatekeeper.
4. Company creates policies in the vendor’s cloud that says “User A can access App X via app/VM/container Z”
5. Even if App X is on the same LAN segment as App Y, App Y is invisible to User A because connectivity to the internal apps happens at L4.
It is an interesting architecture. That being said, ZTNA solutions have their own issues as well (you can probably already spot some based on my explanation above!)
(Note: I worked for a security vendor that sold a ZTNA solution as part of their ~4-5 years ago. Things could be different now.)
what's the difference between this and a reverse SSH tunnel, for example making a local port on your laptop accessible to a public-facing internet server or even running on localhost on that same server... or using sshuttle to access your local network from a remote server .... it doesn't sound like "zero trust" if you're proxying everything through some third-party company that you know nothing about what they're doing with your actual data that you're sending across the wire...
> Now all traffic going to this domain will go through the cloudflared tunnel, which is configured to route homeassistant.mydomain.com to 192.168.1.3. No Warp client needed, Argo tunnel does everything for us.
It boggles my mind that Cloudflare ever considered this acceptable for production, let alone that this is still how tunnels work. The whole configuration scheme feels like something that someone might have kludged up as a technology demo and launched in a staging environment. But the fact that a very security sensitive production system where a “DNS” record that looks like a CNAME to a magic hostname causes traffic to get proxied and sent to a “Zero Trust” private network is just … unreal. It’s almost impossible to tell WTF is going on or what policies apply to what. Does Cloudflare’s proxy really try to fetch an upstream resource, notice that the configured domain name ends with “cfargotunnel.com” and invoke some special handling? What happens if, say, someone else adds that same CNAME to their own network? What if some route goes to foo.bar.com and foo.bar.com’s nameserver reports a CNAME to cfargotunnel.com?
I’ve been using this product for several years, and the documentation and configuration pages have slowly evolved from abysmal to very slightly better. At least now it’s sort of clear how tunnels interact with strict TLS.
The cname is just a normal domain. That DNS entry is a real entry. The CNAME is real. You can go directly to that address too. If someone else knows the cname destination they could go to it or cname their own domain to it literally like any other domain.
The only specially handling is cloud flare has a mapping from subdomain to your private network via it's agent and that's it.
I don't get what's the wrong or complicated about this.
I gave you the benefit of the doubt for a moment, but as far as I can tell, you are incorrect for practical purposes. I went ahead and re-checked everything to make sure. Let's see:
1. I have a cloudflare domain with a working tunnel (managed through Access). In DNS Records, it shows as a CNAME to [redacted].cfargotunnel.com. But:
and no records are returned. Interestingly, it's an empty result, no NXDOMAIN.
2. I have multiple subdomains that appear to be CNAMEs to the same [redacted].cfargotunnel.net. And yet they are entirely different sites that just happen to share an instance of cloudflared at the origin. The sites aren't even served at the same origin address!
They are different "Published Application Routes". They don't even have the same protocol!
2. The tunnel above is on a domain with "Full (strict)" TLS. But traffic to the origin emerges from cloudflared in cleartext.
This whole configuration schema is nonsense. What should happen if a CNAME points at a tunnel that doesn't have a route for that application? What if a tunnel has a route for an application that is CNAMEd somewhere else?
I imagine that what's going on is that Cloudflare internally has a rule that traffic with a cfargotunnel.com origin goes out their Tunnel infrastructure instead of out to the normal Internet. And Cloudflare applies the same JWT that it would apply if the request went out via the normal Internet, and cloudflared verifies that JWT if "Enforce Access JSON Web Token (JWT) validation" is on (maybe the request is literally TLS wrapped inside the cloudflared tunnel? I've never tried to inspect what's going on inside). And then cloudflared unwraps everything? And if you configure cloudflared wrong, then it's totally insecure?
>What happens if, say, someone else adds that same CNAME to their own network?
Cloudflare network carries identity with traffic. If someone else adds the CNAME, they need an identity in the Zero Trust account that controls the tunnel. If you use the browser, Cloudflare IdP MITM the request and requires login to Cloudflare first. If you use Cloudflare Warp, then identity you use to login to Warp is injected.
>CNAME to a magic hostname causes traffic to get proxied and sent to a “Zero Trust” private network
That's also commonly called a load balancer.
From the Cloudflare UI, it works like:
- URL Normalization
- Redirect Rules
- URL Rewrites
- Page Rules
- Configuration Rules
- Origin Rules
- IP Access Rules
- DDoS protection
- Web Application Firewall
- Bots
- Rate Limiting
- Access
- Bulk Redirects
- Modify Request Header
- Cache Rules
- Snippets
- Cloud Connector
- Workers
- Custom Error Rules
- Modify Response Header
- Compression Rules
The "Access" step is key. Cloudflare acts like an authenticating reverse proxy. Once the request is authenticated, it continues processing and can route to the private backend over the Cloudflare tunnel.
Of course, you can make your app public. This is no different security wise than me adding a CNAME my-special-google.my-tld.com to google.com. Whether is works or not depends on the recipient server setup
However, home users live in IPv4 NAT world and look for solutions:
> Expose private services to the public, on public hostnames, no matter where they are running. You could even put your router running at 192.168.1.1 on the internet, accessible to everyone, no Warp client required
Trusting Cloudflare mitm 100% is a means to their goal.
One thing that makes Cloudflare worse for home usage is it acts as a termination point for TLS, whereas Tailscale does not. If you use a Tailscale Funnel, you get the TLS certificate on your endpoint. With Cloudflare, they get a TLS certificate for you, and then strip and optionally re-add TLS as traffic passes through them.
I actually have no idea how private networks with WARP are here, but that's a pretty big privacy downgrade for tunneling from the Internet.
I also consider P2P with relay fallback to be highly desirable over always relaying traffic through a third party, too. Firstly, less middlemen. Secondly, it continues working even if the coordination service is unavailable.
I ended up building something in this space recently (TunnelBuddy – https://www.tunnelbuddy.net I’m the author) that lets you use a friend’s machine as an exit node over WebRTC.
One of the design decisions I made was P2P or nothing: there’s a small signalling service, but no TURN/relay servers. If the peers can’t establish a direct connection, the tunnel just doesn’t come up.
The trade-off is fewer successful connections in weird NAT setups, but in return you know your traffic never transits a third-party relay – it goes straight from your client to your friend’s endpoint.
My traffic will transit third parties all the time, since it's going over the Internet. What's the problem with relays, if the traffic is end-to-end encrypted?
Fair point!
- With a TURN/relay, you’re introducing a single, purpose-built box that: - sees all the tunnel metadata for many users (IP pairs, timing, volume), - is easy to log at or subpoena/compel, - and becomes a natural central chokepoint if someone wants to block the system.
- Without that relay, your traffic still crosses random ISPs/routers, but: - those hops are *generic Internet infrastructure*, not “the TunnelBuddy relay”, - there’s no extra entity whose whole job is to see everyone’s flows.
Zero Trust, except for the trust in Cloudflare.
I generally prefer tailscale and trust them more than cloudflare to not rug-pull me on pricing, but the two features that push me towards cloudflared is the custom domains and client-less access. I could probably set it up with caddy and some plugins, but then I still need to expose the service and port forward.
I'm definitely not trying to dissuade anyone from using Cloudflare, just making sure people realize the potential privacy implications of doing so. It isn't always obvious, even though some of the features pretty much require it (at least to be handled entirely on Cloudflare's side. You could implement similar features that are split between the endpoint and the coordination server without requiring full TLS stripping. Maybe Tailscale will support some of those as features of the `serve` server?)
> client-less access
JFYI, Tailscale Funnels also work for this, though depending on your use case it may not be ideal. Ultimately, Cloudflare does handle this use case a bit better.
Tailscale funnels do work, but it's public only. No auth.
Yeah, because the auth can't be done on Tailscale's end if they don't terminate the TLS connection. However, it is still possible to use an authentication proxy in this situation. Many homelab and small to medium size company setups use OAuth2 Proxy, often with Dex. If you wanted to get fancier, you could use Tailscale for identity when behind the firewall and OAuth2 Proxy when outside the firewall.
This may seem like a lot of effort and it is definitely not nothing, but Cloudflare Tunnels also has a decent number of moving parts and frankly their authentication gateway leaves a bit to be desired for home users.
Tailscale ‘serve’ works well at my startup. SSL and DNS still but unlike funnel it’s limited to your tailscale network.
That's a fair personal decision, but if I would have to put money on it I'd say the chances of new company that raised 160 million of VC funding this year alone vs. established profitable company with a track record of offering free services for many years already I'd put my money on the latter.
> I could probably set it up with caddy and some plugins, but then I still need to expose the service and port forward.
Not so! I have custom domains on my Tailnet with Caddy. You just need to set up Caddy to perform the ACME DNS challenge to get certs for your domain (ironically I use Cloudflare for DNS because they make it very easy to set this up with just an API key). No plugins or open ports needed.
> Cloudflare […] acts as a termination point for TLS
This doesn’t sounds zero-trust at all to me. In fact, it’s as far from zero trust as you can get.
The other option from this great list https://github.com/anderspitman/awesome-tunneling which seems to meet both sets of goals is NetFoundry.
1. End-to-end encryption.
2. Performance and reliability. 100+ PoPs in all major clouds running their data plane routers if they host (still E2EE), or run routers anywhere if you self-host. Dynamic routing to find best paths across the routers.
I don't see any indication that NetFoundry zrok supports end-to-end encryption from the client to the web server. The default configuration definitely terminates SSL on NetFoundry's server, and I don't see any documentation for how to avoid that. There's a TCP tunneling mode, but servers that use this mode can only be accessed by clients that are themselves also connected to the NetFoundry VPN service, not by clients on the public web. What's needed is a TLS tunneling mode that figures out the correct target via SNI, and zrok doesn't seem to provide that.
You are correct, zrok doesn't support mutual TLS. zrok is the free offering that NetFoundry supports so it's easy to see why you looked there for information.
The productized version, NetFoundry Frontdoor (doc here https://netfoundry.io/docs/frontdoor/how-to-guides/create-mt...) is what offers mutual TLS support.
It'll still terminate TLS at the servers, though. It's not mTLS all the way through to the endpoint.
I think the issue is zrok vs. NetFoundry/OpenZiti. Zrok is the easy button to project a public endpoint from inside a network. It is not encrypted all the way through, as it is a proxy solution. NetFoundry/OpenZiti provides methods to provide tunnels all the way through. NetFoundry is a company, OpenZiti is a FOSS project/technology sponsored by NetFoundry, and zrok is a product of NetFoundry built on OpenZiti tech, so it is easy to cross things up. I think the comment was in regard to NetFoundry/OpenZiti, while your response referenced zrok. The list above has both.
i should have been more clear - you have the option:
+ e2ee via netfoundry's zero trust products
+ non-e2ee via netfoundry frontdoor
Tunneling p2p with relay fallback is essentially what connet [1] aspires to be. There are a lot of privacy/security benefits exposing endpoints only to participating peers. You can either run it yourself or use hosted version [2].
[1] https://github.com/connet-dev/connet
[2] https://connet.dev
TLS termination is neither required nor enabled by default, right?
For tunnels many of the features basically have to work this way, so I'd be surprised if you could avoid it. It's also impossible to avoid if you use normal Cloudflare "protected" DNS entries. You can use Cloudflare as just a DNS server but it's not the default, by default it will proxy everything through Cloudflare, since that's kind of the point. You can't cache HTTP requests you can't see.
Correct. We run it without it and just use the DNS filtering aspect.
How does it do DNS filtering without TLS interception - takeover for DNS resolution?
In what way are DNS resolution and TLS related except for the little-used DoT?
Thats a big privacy issue if they strip TLS, does it have a technical reason or they just don't want to offer privacy?
Is it technically possible to have something like Tailscale funnel but with something like Cloudflare Access authentication (at least for some options)?
That would be great!!
For that kind of end-to-end encryption I use pinggy.io tls tunnels.
What's the actual win here? Avoiding relay latency in the rare cases Tailscale can't punch through NAT? If that's it, a $3 VPS running Headscale seems simpler. The complexity feels like you're optimizing for the 5% case while adding permanent vendor lock in. What am I missing?
Tailscale has what they call Peer Relays now to help solve this problem:
https://tailscale.com/blog/peer-relays-beta
For many homelabbers, just being cheap and avoiding the $3 VPS, that's it
Exactly, just today I set up a cloudflare tunnel to a docker compose service running on my home server. I didn’t want to expose the server directly to the internet, and I want to share this service on a certain domain with broader family.
I have a server at home that works well. I don’t reaaaally want to pay an extra $30-$40/yr and have an extra thing to manage when the CF tunnel works fine for free. I like Tailscale more, but I want to share this with family who won’t install TS and also want to use a specific domain.
I dont even pay anything, my tiny homelab is completely covered by the free tier
What provider still has decent free tier?
Oracle. 4 vCPU, 24 GB RAM, 200 GB SSD. It’s arm64 but nowadays that doesn’t really matter.
That can't possibly be free?
https://docs.oracle.com/en-us/iaas/Content/FreeTier/freetier...
Only caveat I see is they reserve the right to delete underutilized/ idling instances
I guess they require a credit card before accessing that free tier?
If they do: create a virtual one, create an account with it and delete the card right after.
That’s generous but Oracle is very generous.
Which region were you able to create this in? They seem to be out of capacity all the time in EU.
What worked for me was handing them a credit card and transitioning myself out of the free tier. (I'd use the free credits they offer prior to doing this - they give you something like $300 immediately on signup.)
The always-free infra remains free, you just have the chance of incurring a bill if you make selections that aren't free or exceed block storage/egress (200GB/10TB) limits of the always-free tier. Leaving the free/trial tier gives you access to a much larger pool of instances. I never successfully deployed an A1 instance prior to becoming a "paying" customer - now I've done it hundreds of times without ever having an issue.
I've been running a small k0s cluster and a standalone webserver for months while incurring about $2.50 - $3 in spending each month, primarily from being slow to remove instance snapshots sitting in block storage.
Even things that are oddly expensive on AWS - like NAT - are free on Oracle. There are zero gotchas.
I hit the same roadblock as the above user and it never occurred to me to just cross the barrier with cash and then scale back to free. Thanks for this.
It doesn't actually charge you anything. You just have to put a card down to be considered a priority because now you potentially can spend money & therefore are more important then the other free-tier losers. /s It's still free tier & still free.
The free tier is also based on capcity usage, and not instances. If you want 3 cores on 1 machine & 1 on another, they're cool with that. I personally run Pangolin on a 1 core & self-hosted github runners on a 3 core.
I have read that you need to write a script to constantly bombard their API in order to get one. I presume you'd be fighting other scripts.
$3 VPS running Headscale is not simpler since you won't be able to run both headscale and tailscale on your end user machines, I don't recommend it.
The solution we've found is running a white IP container (or VPS) which looks like regular Wireguard outside, while inside it "forwards" to your existing tailscale network.
I don't remember if we use https://github.com/gravitl/netmaker or https://github.com/juhovh/tailguard
Also see: https://tailscale.com/blog/peer-relays-beta
What does Headscale have to do with NAT hole punching? I believe what you actually mean is setting up a relay, see the link in https://news.ycombinator.com/item?id=45948806 .
I don't think you are missing anything. They have a bunch of half baked features like this that aren't as robust as real security vendors and lock you in just like you said.
Maybe I’m misunderstanding something…
But are you accusing someone of promoting vendor lock-in (cloudflare) while at the same time promoting vendor lock-in (tailscale)?
If you’re ok with vendor lock-in, shouldn’t you in theory be ok with any vendor?
Headscale is the not-vendor-login version of Tailscale.
Sort of. Many tailscale clients you would use with headscale are closed source.
No. Not all vendors are equal. We can treat ProtonMail differently then Gmail, for example. Looking at what's gone down with VMware, definitely don't get in bed with Broadcom.
> after frustration with Tailscale in environments where it couldn’t properly penetrate NAT/firewall and get a p2p connection, I decided to invest some time into learning something new: Cloudflare Zero Trust + Warp
...which doesn't even try to get a p2p connection. Instead you always get the thing you didn't want. If you're okay with that you could've just ignored how Tailscale connected those devices, that's kind of the point. You've also in the process converted your entire security model to Cloudflare's idea of "Zero Trust" which involves 100% trusting Cloudflare.
The rest of the blog post is fine, but the motivation is honestly baffling.
The difference is that Cloudflare has their own high quality network and PoPs everywhere, so the quality is generally even better than P2P.
This is my experience, we are a fully remote world-wide company and we recently migrated away from Tailscale to Cloudflare and it has been much better.
Eh, kind of.
Peering in Europe is such a mess that even Cloudflare can be pretty bad. Sometimes you have to manually calculate "okay, there's a colo in this particular city that will force the correct route if we proxy all our traffic through it ..."
Is the connection through Cloudflare still encrypted between the two peers, as it would be going through a Tailscale relay? If not, that's definitely a downgrade using the Cloudflare approach. But if not, I'm not sure the trust model is significantly different with maybe the added benefit of the fact that Cloudflare's relay performance is likely better given that relaying traffic is kind of their main thing rather than a very secondary function like it is for Tailscale.
On the other hand, my experience with Tailscale is that they're very, very good at NAT hole punching and I'd rather have a direct connection where possible from a latency standpoint.
I've experimented with Cloudflare tunnels before to sit in front of my Immich instance in my homelab. Only issue is the 100MB upload size for videos. But Immich added upload chunking support to their roadmap so its possible this will work very well in the future.
Immich also has the ability to use different domains for different networks. Meaning that I connect directly to my server when I'm connected to my local home network and connect through cloudflare when I'm out of my house.
This way I can upload big videos when I get home.
With that it becomes clear that some service is self hosted (the DNS record points to a private IP). It can be a security issue when the Whois record or the domain name allows the identification of the hosting entity. Finding its physical address can be an easy task depending on its social presence.
Then probably the hosting place is an easier target than a data center.
This seems like an excellent guide. I love these "how the pieces fit together" kind of guides.
Perhaps CF could license it and slap it in their docs!
Nice article. For easily exposing private services to the internet I’ve been using https://tuns.sh which lets you run ssh tunnels. It’s nice for a zero install solution.
Free Cloudflare account cannot be used to serve my Plex server. To me that's a no-go.
The specific term is: https://www.cloudflare.com/service-specific-terms-applicatio...
My free account runs cloudflared tunnel just fine for jellyfin. Is it just plex?
I use the tunnel because my girlfriend cant install tailscale on her work laptop, so this way she can still login to jellyfin while traveling.
It’s not that it can’t do it, it’s that it’s against the terms of service to use it for video.
Isn’t that for their CDN, rather than Zero Trust? I don’t think they even know what traffic is going over Zero Trust.
> it’s against the terms of service to use it for video
It for any large files. They want to limit bandwidth usage but not blanket limit everyone. One user using Plex of Jellyfin probably doesn't move the needle.
You can't play corporate videos over Cloudflare Warp? Where is the line drawn?
I think they don't want you using a boatload of bandwidth, since data proxies through their servers.
Yes and that's what people are criticizing - it's just an arbitrary and thus very bad rule. Completely unrelated to CloudFlare, I streamed a single TV series from a friend's plex account 1-2y ago, that's less traffic than some of my friends use in 2 weeks.
I'm not saying they can't have that rule, it's their infra - I'm just saying that "a boatload of bandwidth" can be anything, depending on who you ask.
FWIW, even if it’s against the rules I have been using Cloudflare tunnels for my Jellyfin server for months and I haven’t even gotten a nastygram yet.
I think it’s more of a “we reserve the right to…” situation than anything else.
Why would anyone expect anyone else to serve video for them for free? Is there any CDN that serves video for you for free???
I'm not even talking about the copyright implications here, just the bandwidth costs. A single movie download would cost more than many hundreds of typical simple HTTP website sessions.
> Why would anyone expect anyone else to serve video for them for free?
I would expect that a freemium service selling encrypted "zero trust" networking should have no idea what traffic is being pushed through my network making enforcement impossible.
Nobody's asking for a free lunch, but the reasonable thing to do would be to simply bandwidth limit freemium accounts across the board, not make exceptions for certain kinds of traffic in what should be a secure network.
But those are different services. You linked the CDN TOS.
(I can, in practice, serve small amounts of video through the free CDN, by using HLS; I'm also aware that this needs to stay below the radar)
I didn’t link anything.
Cloudflare does say "video and other large files" so in the end it is about volume, not data type. They probably just want to have the arbitrary decision on specific cases without defining a uniform blanket limit.
It’s not enforced in practice unless you run a giant server with a lot of traffic.
Not sure if you are a representative from Cloudflare. But regardless, Cloudflare intentionally made it ambiguous so that they can selectively enforce it.
For sure. But OPs point still stands, people run giant Plex servers and charge for access to the pirated content on them. Huge numbers of users. I imagine they want to enforce on those folks and not people who just happen to have their own running.
Agreed, I have been doing this for years without consequences so far
You can use Cloudflare for DNS, but you just can’t use them for proxying the traffic. I use Cloudflare with proxying on for everything other than a plex subdomain.
You could just disable cdn/caching.
Do you have IPv6 from your ISP?
I use IPv6 only Emby and Jellyfin servers with my friends. We all have native IPv6 and it works well for us.
i use it for my jellyfin and it just works, been months like this
Big fan of Cloudflare Tunnel here, too.
We use our Windows workstations as WSL SSH tunnels, protected with email verification (only for our domain), and it’s been working perfectly.
I’m curious, though, about how we can expose Docker services. It would be fantastic to have a remote build server set up with Cloudflare Tunnel.
I'm using Netbird [0] for my home / private needs: - Synology NAS - All the laptops and desktops my family uses - All family mobile phones
Given i work in Tmux, its super convenient to take a laptop with me and just use it as a thin client to my Desktop wherever I am.
[0] https://netbird.io/
Thanks a lot. Both the post itself and the comments are very useful. I'd like to comment on the content of this post, but it'll take much time for me to wrap my head around.
In the mean time: the images in the article seem to be broken — produce 404 errors. Like this one: https://david.coffee/targets-config-screen.png
Tailscale now has the awesome feature of peer relays and now there's no more excuses why you can't traverse that NAT and you can forget about all those DERP servers.
But is this vendor-lockin, as in CloudFlare being the vendor here? Because at least with Tailscale there's no vendor lock-in.
Actually, seems TailScale is also a vendor? huh, and I thought it was an opensource project...
Tailscale is almost open source with Wireguard itself being open source.
- Most of the clients are open source probably.
- Tailscale allows you to run custom control server of your own.
- One open source control server "headscale" is sponsored by Tailscale themselves.
Oh man, someone has to talk about this!! Cory told me about CF's gold issues and it really does seem problematic to me, I'm glad ZT is finally being criticised.
I stopped reading at "everything goes through the Cloudflare network, no direct p2p"
https://github.com/alecbcs/hyprspace has penetrated every NAT I've ever encountered. No megacorporation required.
That project appears abandoned and unmaintained.
I don't understand the use case here. Is this supposed to be for enterprise to control access to internal applications via network access policies?
Yes. The acronym is “ZTNA” (Zero Trust Network Access).
It is an alternative to a traditional corporate VPN that addresses a few architectural issues; namely:
- L3 connectivity (which allows for lateral movement) to the corporate network. - Inbound exposure to the VPN gateway (scaling can become a challenge, not to mention continuous vulnerabilities from… certain vendors) - Policy management can get convoluted if you want to do micro-segmentation properly.
ZTNA is essentially an “inside-out” architecture and acts (kind of) like a L4 proxy. I’m going to butcher this explanation, but:
1. Company installs apps/VMs/containers throughout their network. These must have network reachability to the internal apps/services the company wants to make available to its users.
2. These apps/VMs/containers establish TLS tunnels back to the company’s tenant in the vendor’s cloud.
3. Company rolls out the vendor’s ZTNA client to user devices. This also establishes a TLS tunnel to the vendor’s cloud. Hence the vendor’s cloud is like a MitM gatekeeper.
4. Company creates policies in the vendor’s cloud that says “User A can access App X via app/VM/container Z”
5. Even if App X is on the same LAN segment as App Y, App Y is invisible to User A because connectivity to the internal apps happens at L4.
It is an interesting architecture. That being said, ZTNA solutions have their own issues as well (you can probably already spot some based on my explanation above!)
(Note: I worked for a security vendor that sold a ZTNA solution as part of their ~4-5 years ago. Things could be different now.)
Yes, this is exactly what this does.
what's the difference between this and a reverse SSH tunnel, for example making a local port on your laptop accessible to a public-facing internet server or even running on localhost on that same server... or using sshuttle to access your local network from a remote server .... it doesn't sound like "zero trust" if you're proxying everything through some third-party company that you know nothing about what they're doing with your actual data that you're sending across the wire...
Zero trust is a marketing term used by them - surprisingly it has nothing to do with end-to-end encryption also.
[flagged]
> Then go into Cloudflare DNS settings and map the domain homeassistant.mydomain.com to the tunnel:
> CNAME homeassistant.mydomain.com a2f17e27-cd4d-4fcd-b02a-63839f57a96f.cfargotunnel.com
> Now all traffic going to this domain will go through the cloudflared tunnel, which is configured to route homeassistant.mydomain.com to 192.168.1.3. No Warp client needed, Argo tunnel does everything for us.
It boggles my mind that Cloudflare ever considered this acceptable for production, let alone that this is still how tunnels work. The whole configuration scheme feels like something that someone might have kludged up as a technology demo and launched in a staging environment. But the fact that a very security sensitive production system where a “DNS” record that looks like a CNAME to a magic hostname causes traffic to get proxied and sent to a “Zero Trust” private network is just … unreal. It’s almost impossible to tell WTF is going on or what policies apply to what. Does Cloudflare’s proxy really try to fetch an upstream resource, notice that the configured domain name ends with “cfargotunnel.com” and invoke some special handling? What happens if, say, someone else adds that same CNAME to their own network? What if some route goes to foo.bar.com and foo.bar.com’s nameserver reports a CNAME to cfargotunnel.com?
I’ve been using this product for several years, and the documentation and configuration pages have slowly evolved from abysmal to very slightly better. At least now it’s sort of clear how tunnels interact with strict TLS.
The cname is just a normal domain. That DNS entry is a real entry. The CNAME is real. You can go directly to that address too. If someone else knows the cname destination they could go to it or cname their own domain to it literally like any other domain.
The only specially handling is cloud flare has a mapping from subdomain to your private network via it's agent and that's it.
I don't get what's the wrong or complicated about this.
I gave you the benefit of the doubt for a moment, but as far as I can tell, you are incorrect for practical purposes. I went ahead and re-checked everything to make sure. Let's see:
1. I have a cloudflare domain with a working tunnel (managed through Access). In DNS Records, it shows as a CNAME to [redacted].cfargotunnel.com. But:
$ dig [redacted].cfargotunnel.com
; <<>> DiG 9.10.6 <<>> [redacted].cfargotunnel.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5851 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
and no records are returned. Interestingly, it's an empty result, no NXDOMAIN.
2. I have multiple subdomains that appear to be CNAMEs to the same [redacted].cfargotunnel.net. And yet they are entirely different sites that just happen to share an instance of cloudflared at the origin. The sites aren't even served at the same origin address!
They are different "Published Application Routes". They don't even have the same protocol!
2. The tunnel above is on a domain with "Full (strict)" TLS. But traffic to the origin emerges from cloudflared in cleartext.
This whole configuration schema is nonsense. What should happen if a CNAME points at a tunnel that doesn't have a route for that application? What if a tunnel has a route for an application that is CNAMEd somewhere else?
I imagine that what's going on is that Cloudflare internally has a rule that traffic with a cfargotunnel.com origin goes out their Tunnel infrastructure instead of out to the normal Internet. And Cloudflare applies the same JWT that it would apply if the request went out via the normal Internet, and cloudflared verifies that JWT if "Enforce Access JSON Web Token (JWT) validation" is on (maybe the request is literally TLS wrapped inside the cloudflared tunnel? I've never tried to inspect what's going on inside). And then cloudflared unwraps everything? And if you configure cloudflared wrong, then it's totally insecure?
>What happens if, say, someone else adds that same CNAME to their own network?
Cloudflare network carries identity with traffic. If someone else adds the CNAME, they need an identity in the Zero Trust account that controls the tunnel. If you use the browser, Cloudflare IdP MITM the request and requires login to Cloudflare first. If you use Cloudflare Warp, then identity you use to login to Warp is injected.
>CNAME to a magic hostname causes traffic to get proxied and sent to a “Zero Trust” private network
That's also commonly called a load balancer.
From the Cloudflare UI, it works like:
- URL Normalization
- Redirect Rules
- URL Rewrites
- Page Rules
- Configuration Rules
- Origin Rules
- IP Access Rules
- DDoS protection
- Web Application Firewall
- Bots
- Rate Limiting
- Access
- Bulk Redirects
- Modify Request Header
- Cache Rules
- Snippets
- Cloud Connector
- Workers
- Custom Error Rules
- Modify Response Header
- Compression Rules
The "Access" step is key. Cloudflare acts like an authenticating reverse proxy. Once the request is authenticated, it continues processing and can route to the private backend over the Cloudflare tunnel.
Of course, you can make your app public. This is no different security wise than me adding a CNAME my-special-google.my-tld.com to google.com. Whether is works or not depends on the recipient server setup
This is all true.
However, home users live in IPv4 NAT world and look for solutions:
> Expose private services to the public, on public hostnames, no matter where they are running. You could even put your router running at 192.168.1.1 on the internet, accessible to everyone, no Warp client required
Trusting Cloudflare mitm 100% is a means to their goal.