Use lighttpd to access a website over a proxy

mf6190 asked:

this is going to be my very first question.

At our company we are using Squid as a proxy between our intranet and the internet. Now we have the need that a software package can’t handle a proxy server – it wants direct internet access.

So my idea was to use lighttpd as a “bridge” between this application and the internet. So that the application can call “mylighttpd:91234” and see the Google page for example – with the address in the browser still being “mylighttpd:91234”.

This is how my lighttpd.conf looks like:

config {
var.PID                        = 13793
var.CWD                        = "/home/testusr"
var.log_root                   = "/tmp/lighttpd-log"
var.server_root                = "/tmp/lighttpd-data"
var.state_dir                  = "/tmp/lighttpd-var/run"
var.home_dir                   = "/var/lib/lighttpd"
var.conf_dir                   = "/etc/lighttpd"
var.vhosts_dir                 = "/tmp/lighttpd-data/vhosts"
var.cache_dir                  = "/var/cache/lighttpd"
var.socket_dir                 = "/var/lib/lighttpd/sockets"
server.modules                 = (
    "mod_indexfile",
    "mod_access",
    "mod_proxy",
    "mod_status",
    "mod_accesslog",
    "mod_redirect",
    "mod_rewrite",
    "mod_accesslog",
    "mod_dirlisting",
    "mod_staticfile",
    # 10
)
server.port                    = 91234
server.use-ipv6                = "disable"
server.username                = "testusr"
server.groupname               = "testgroup"
server.document-root           = "/tmp/lighttpd-data/htdocs"
server.pid-file                = "/tmp/lighttpd-var/run/lighttpd.pid"
server.errorlog                = "/tmp/lighttpd-log/error.log"
accesslog.filename             = "/tmp/lighttpd-log/access.log"
server.event-handler           = "linux-sysepoll"
server.network-backend         = "linux-sendfile"
server.max-fds                 = 2048
server.stat-cache-engine       = "simple"
server.max-connections         = 1024
index-file.names               = ("index.xhtml", "index.html", "index.htm", "default.htm", "index.php")
url.access-deny                = ("~", ".inc")
url.redirect                   = (
    "^/(.*)" => "http://google.com/$1",
)
proxy.debug                    = 1
proxy.server                   = (
    "" => (
        (
            "host" => "192.168.1.10",
            "port" => 8080,
            # 2
        ),
    ),
)
static-file.exclude-extensions = (".php", ".pl", ".fcgi", ".scgi")
mimetype.use-xattr             = "disable"
mimetype.assign                = (
    ".pdf"     => "application/pdf",
    ".sig"     => "application/pgp-signature",
    ".spl"     => "application/futuresplash",
    ".class"   => "application/octet-stream",
    ".ps"      => "application/postscript",
    # 5
    ".torrent" => "application/x-bittorrent",
    ".dvi"     => "application/x-dvi",
    ".gz"      => "application/x-gzip",
    ".pac"     => "application/x-ns-proxy-autoconfig",
    ".swf"     => "application/x-shockwave-flash",
    # 10
    ".tar.gz"  => "application/x-tgz",
    ".tgz"     => "application/x-tgz",
    ".tar"     => "application/x-tar",
    ".zip"     => "application/zip",
    ".mp3"     => "audio/mpeg",
    # 15
    ".m3u"     => "audio/x-mpegurl",
    ".wma"     => "audio/x-ms-wma",
    ".wax"     => "audio/x-ms-wax",
    ".ogg"     => "application/ogg",
    ".wav"     => "audio/x-wav",
    # 20
    ".gif"     => "image/gif",
    ".jpg"     => "image/jpeg",
    ".jpeg"    => "image/jpeg",
    ".png"     => "image/png",
    ".xbm"     => "image/x-xbitmap",
    # 25
    ".xpm"     => "image/x-xpixmap",
    ".xwd"     => "image/x-xwindowdump",
    ".css"     => "text/css",
    ".html"    => "text/html",
    ".htm"     => "text/html",
    # 30
    ".js"      => "text/javascript",
    ".asc"     => "text/plain",
    ".c"       => "text/plain",
    ".cpp"     => "text/plain",
    ".log"     => "text/plain",
    # 35
    ".conf"    => "text/plain",
    ".text"    => "text/plain",
    ".txt"     => "text/plain",
    ".spec"    => "text/plain",
    ".dtd"     => "text/xml",
    # 40
    ".xml"     => "text/xml",
    ".mpeg"    => "video/mpeg",
    ".mpg"     => "video/mpeg",
    ".mov"     => "video/quicktime",
    ".qt"      => "video/quicktime",
    # 45
    ".avi"     => "video/x-msvideo",
    ".asf"     => "video/x-ms-asf",
    ".asx"     => "video/x-ms-asf",
    ".wmv"     => "video/x-ms-wmv",
    ".bz2"     => "application/x-bzip",
    # 50
    ".tbz"     => "application/x-bzip-compressed-tar",
    ".tar.bz2" => "application/x-bzip-compressed-tar",
    ".rpm"     => "application/x-rpm",
    ""         => "application/octet-stream",
    # 54
)
dir-listing.activate           = "disable"
dir-listing.hide-dotfiles      = "disable"
dir-listing.exclude            = ("~$")
dir-listing.encoding           = "UTF-8"
dir-listing.hide-header-file   = "disable"
dir-listing.show-header        = "disable"
dir-listing.hide-readme-file   = "disable"
dir-listing.show-readme        = "disable"
server.follow-symlink          = "enable"
server.upload-dirs             = ("/var/tmp")


$HTTP["url"] =~ ".pdf$" {
    # block 1
    server.range-requests = "disable"

} # end of $HTTP["url"] =~ ".pdf$"
}

Whereas:
* 91234: the port is where lighttpd listens on
* 192.168.1.10: the IP of the Squid proxy
* www.google.de: the page lighttpd should forward to via the proxy.

At the moment, it opens the Google page when I type “localhost:9123” in my browser, but it replaces the address with www.google.com, where it should stay at “localhost:9123”.

I’ve read the docs of lighttpd and Apache about forwarding, redirection and proxies, but I can’t say it differently – it simply doesn’t go into my head.

Thanks for your help and understanding.

My answer:


Just set up interception, redirecting all outbound requests from that server to squid. This is non-trivial, and you might even need to set up a second squid server for it, but it’s a lot less messy than the above proposal, and it even uses tools you’re already familiar with.


View the full question and answer on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.