.

Donate to FreeBSD

.

Home

About

Introduction

Features

Privacy Policy

Projects

Ports

Get FreeBSD

Release Information

Release Engineering

Documentation

FAQ

Handbook

Porter's Handbook

Developer's Handbook

Manual Pages

Presentations and Papers

Documentation Project Primer

All Books and Articles

Community

Mailing Lists

Forums

User Groups

Events

FreeBSD Journal

Q&A (external)

Developers

Project Ideas

Git Repository

GitHub Mirror

Code Review (Phabricator)

Wiki

Continuous Integration Service

Support

Vendors

Security Information

Bug Reports

Submitting Bug Reports

Foundation

Monetary Donations

Hardware Donations

Forums

New posts

Search forums

What's new

New posts

Latest activity

Log in

Register

What's new

Search

Search

Everywhere
Threads
This forum
This thread

Search titles only

By:

Search
Advanced search…

.

Donate to FreeBSD

.

New posts

Search forums

Menu

Log in

Register

Install the app

Install

Server and Networking

Web and Network Services

Nginx pkg cache help

Thread starter

Blue|Fusion

Start date

Jul 1, 2022

B

Blue|Fusion

Jul 1, 2022

#1

Hello all,

I am setting up an Nginx caching web server in a FreeBSD 13.1 Jail. It is working as expected for freebsd-update and my Linux distro of choice, but not for FreeBSD binary packages. The access log shows it is a cache miss and the file is retrieved downstream successfully through the Nginx cache, but the file is never cached - /var/log/nginx/fbsd-pkg remains an empty directory (with proper permissions created at Nginx service start).

Bash:

bf@repocache:~ % sudo du -h -d1 /var/cache/nginx
3.1G /var/cache/nginx/void-repo
12K /var/cache/nginx/fbsd-update
512B /var/cache/nginx/fbsd-pkg
3.1G /var/cache/nginx

Here is my /usr/local/etc/nginx/nginx.conf (my FQDN replaced with example.lan):

(Also open to other recommendations to make this pieced together config more appropriate for the task)

NGINX:

events {
worker_connections 1024;
multi_accept on;
}

http {
aio threads;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;

include mime.types;
default_type application/octet-stream;

log_format cachelog '$remote_addr / $http_x_forwarded_for - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$upstream_cache_status" "$http_range"';

proxy_cache_key "$scheme$request_method$host$request_uri";
proxy_cache_path /var/cache/nginx/fbsd-update levels=1:2 keys_zone=fbsdupdate_cache:10m
max_size=5G inactive=7d use_temp_path=off;
proxy_cache_path /var/cache/nginx/fbsd-pkg levels=1:2 keys_zone=fbsdpkg_cache:10m
max_size=10G inactive=365d use_temp_path=off;
proxy_cache_path /var/cache/nginx/void-repo levels=1:2 keys_zone=voidrepo_cache:10m
max_size=10G inactive=365d use_temp_path=off;

server {
listen 80;
server_name fbsd-update.example.lan;

root /var/cache/nginx/fbsd-update;

access_log /var/log/nginx/fbsd-update-access.log cachelog;

location /nginx-status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow ::1;
deny all;
}

location / {
proxy_cache fbsdupdate_cache;
proxy_cache_lock on;
proxy_buffering on;
proxy_cache_lock_age 5m;
proxy_cache_lock_timeout 15m;
proxy_http_version 1.1;
proxy_cache_revalidate on;
proxy_cache_valid 200 7d;
expires max;
add_header X-Proxy-Cache $upstream_cache_status;

proxy_pass http://update.freebsd.org;

location ~* (latest.ssl)$ {
proxy_cache_bypass 1;
proxy_no_cache 1;
proxy_pass http://update.freebsd.org$request_uri;
}
}
}

server {
listen 80;
server_name fbsd-pkg.example.lan;

root /var/cache/nginx/fbsd-pkg;

access_log /var/log/nginx/fbsd-pkg-access.log cachelog;

location /nginx-status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow ::1;
deny all;
}

location / {
proxy_cache fbsdpkg_cache;
proxy_cache_lock on;
proxy_cache_lock_age 5m;
proxy_cache_lock_timeout 15m;
proxy_buffering on;
proxy_http_version 1.1;
proxy_cache_revalidate on;
proxy_cache_valid 200 365d;
expires max;
add_header X-Proxy-Cache $upstream_cache_status;

proxy_pass https://pkg.freebsd.org;

location ~* (packagesite.pkg)$ {
proxy_cache_bypass 1;
proxy_no_cache 1;
proxy_pass https://pkg.freebsd.org$request_uri;
}
location ~* (packagesite.txz)$ {
proxy_cache_bypass 1;
proxy_no_cache 1;
proxy_pass https://pkg.freebsd.org$request_uri;
}
location ~* (meta.conf)$ {
proxy_cache_bypass 1;
proxy_no_cache 1;
proxy_pass https://pkg.freebsd.org$request_uri;
}
location ~* (meta.txz)$ {
proxy_cache_bypass 1;
proxy_no_cache 1;
proxy_pass https://pkg.freebsd.org$request_uri;
}
}
}

server {
listen 80;
server_name void-repo.example.lan;

root /var/cache/nginx/void-repo;

access_log /var/log/nginx/void-repo-access.log cachelog;

location /nginx-status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow ::1;
deny all;
}

location / {
proxy_cache voidrepo_cache;
proxy_cache_lock on;
proxy_buffering on;
proxy_cache_lock_age 5m;
proxy_cache_lock_timeout 15m;
proxy_http_version 1.1;
proxy_cache_revalidate on;
proxy_cache_valid 200 365d;
expires max;
add_header X-Proxy-Cache $upstream_cache_status;

proxy_pass https://repo-us.voidlinux.org;

location ~* (x86_64-repodata)$ {
proxy_cache_bypass 1;
proxy_no_cache 1;
proxy_pass https://repo-us.voidlinux.org$request_uri;
}
}
}
}

OP

B

Blue|Fusion

Jul 2, 2022

Thread Starter

#2

I figured it out. It's not a fault of the configuration but the upstream headers setting Cache-Control: private. I'm not sure why this is, perhaps there's a good reason?

In the mean time, I set Nginx to ignore this header and the packages are now caching locally which is very useful for my 30-something jails.

You must log in or register to reply here.

Share:

Reddit

Pinterest

Tumblr

WhatsApp

Email

Share

Link

Server and Networking

Web and Network Services

FreeBSD Style

Contact us

Terms and rules

Privacy policy

Help

RSS

Community platform by XenForo® © 2010-2023 XenForo Ltd.

The mark FreeBSD is a registered trademark of The FreeBSD Foundation and is used by The FreeBSD Project with the permission of The FreeBSD Foundation.

Back

Top