Outbound HTTP

A WinterCMS plugin fetches a remote URL with the Laravel HTTP client or Guzzle, and checks the URL by hand when it came from a user. When a URL comes from outside the application, such as a remote image address, fetch it with fetchguard. It is the framework's guard against server-side request forgery: a request that a user can aim at the application's own network, a cloud metadata service or an internal admin panel.

For calls to services the application itself chose, such as a payment provider's API, the standard net/http client is fine.

Policies#

Every call takes a fetchguard.Policy:

• fetchguard.AllowHostsMode allows only the hosts in AllowHosts, matched exactly or as a dotted suffix.
• fetchguard.PublicOnlyMode allows any public host.

In both modes only https is allowed, and the resolved IP address is checked when the connection is dialled, so a DNS name that resolves into the network is refused too. The check covers private, loopback, link-local, carrier-grade NAT, documentation, multicast and other reserved IPv4 and IPv6 ranges, including IPv4 addresses inside NAT64 and 6to4 addresses. Environment proxy settings are ignored, so the check always sees the real target.

ctx := context.Background()
// Only the application's image host, at most 5 MiB within 5 seconds.
images := fetchguard.Policy{
	Mode:       fetchguard.AllowHostsMode,
	AllowHosts: []string{"images.example.com"},
	MaxBytes:   5 << 20,
	Timeout:    5 * time.Second,
}
// Any public host, for a URL a user pasted.
public := fetchguard.Policy{Mode: fetchguard.PublicOnlyMode}

for _, c := range []struct {
	url    string
	policy fetchguard.Policy
}{
	{"http://images.example.com/cover.jpg", images},
	{"https://cdn.attacker.example/cover.jpg", images},
	{"https://127.0.0.1/admin", public},
	{"https://169.254.169.254/latest/meta-data/", public},
	{"https://[::ffff:10.0.0.1]/", public},
	{"https://%zz", public},
} {
	// The last argument is the application's config (app.Config), for
	// limits the policy leaves at zero; nil uses the framework defaults.
	_, err := fetchguard.Fetch(ctx, c.url, c.policy, nil)
	var fe *fetchguard.Error
	if errors.As(err, &fe) {
		fmt.Println(fe.Reason, c.url)
	}
}
fmt.Println(fetchguard.Defaults())
// Output:
// scheme http://images.example.com/cover.jpg
// invalid_url https://cdn.attacker.example/cover.jpg
// private_ip https://127.0.0.1/admin
// private_ip https://169.254.169.254/latest/meta-data/
// private_ip https://[::ffff:10.0.0.1]/
// invalid_url https://%zz
// 10485760 10s

A failure is always a fetchguard.Error with one fetchguard.Reason from a closed set, so a handler can map it to a stable API error code. A host outside the allow list is reported as invalid_url, as the example shows.

Responses and limits#

fetchguard.Fetch returns a fetchguard.Result with the body, the content type and the status code for any response the server completed, including 4xx and 5xx. Check the status yourself.

Redirects are never followed: a 3xx response is returned as a result. To follow it, call fetchguard.Fetch again with the Location URL, which runs every check again.

The body is capped at the policy's MaxBytes (a larger body is fetchguard.ReasonTooLarge) and the call at its Timeout. A limit left at zero falls back to http.fetch.max_bytes and http.fetch.timeout_seconds from the configuration you pass, then to the framework defaults of 10 MiB and 10 seconds (fetchguard.Defaults). A configured value of zero or less is an error, not a way to turn a limit off.