Add Support for sniproxy and dnsmasq wildcard domains

.gitignore

@@ -54,6 +54,6 @@ docs/_build/
 # PyBuilder
 target/
 
-config.json
+
 
 *.conf

README.md

@@ -1,52 +1,67 @@
 dnsproxy
 ========
 
-This project is still in beta! It currently works well, but some changes may be expected.
+Project Based on https://github.com/jamiees2/dnsproxy
 
-This is based on @trick77's original work on [tunlr-style-dns-unblocking](https://github.com/trick77/tunlr-style-dns-unblocking/)
+Added sniproxy and domain list to route specific Domains to the proxy.
 
-The purpose for this project is to make it easy to set up a fast smart DNS service on your own. It is possible to get a US VPS for 1$/mo which is well under the current 4$/mo for most smart DNS services. It is also potentially alot faster, as you then have a whole proxy server to yourself.
+Unblock:
 
-Another purpose is privacy. By changing your DNS servers, you allow whoever is on the other side to see almost all the websites you visit. Some providers explicitly state in their TOS that they log everything, which gives them control over your data. A better option is that you control those servers.
+Zattoo
+
+Wilmaa
+
+Teleboy
+
+Blue TV (Swisscom)
+
+....and many more.
 
 Prerequisites:
-- A VPS based in the country you want, preferrably running Ubuntu 14.04. A 128MB server is enough.
+- A VPS based in the country you want, preferrably running Ubuntu 20.04. A 128MB server is enough.
 - python
-- haproxy
-
-For `sni` and `dnat` setup:
+- sniproxy
 - dnsmasq
 
 
-The configuration generator (dnsproxy.py) offers three different possibilities for setup:
-- [sni (Simple Setup)](https://github.com/jamiees2/dnsproxy/wiki/SNI-Setup)
-- [dnat (Advanced Setup)](https://github.com/jamiees2/dnsproxy/wiki/DNAT-Setup)
-- [local (Advanced Setup)](https://github.com/jamiees2/dnsproxy/wiki/Local-Setup)
-- manual (Default)
+The configuration generator (dnsproxy.py) offers works with sniproxy with the following Setup:
+- [sni (Simple Setup)](https://github.com/jamiees2/dnsproxy/wiki/SNI-Setup) (You dont need HAProxy, as we use sniproxy in our project Fork)
+
+
+You can generate each configuration file separately with `-o`. Example:
+
+```python dnsproxy.py -o sniproxy```
+
+```python dnsproxy.py -o dnsmasq```
+
+You can also generate a Hosts file for your Client, then you can skip dnsmasq. 
+
+```python dnsproxy.py -o hosts```
+
 
+Copy and overwrite the generated configs for sniproxy and dnsmasq.
+Restart the Service and your dns Proxy should be up and running.
 
-You can generate each configuration file separately with `-m manual`. Example:
-```python dnsproxy.py -m manual -o haproxy```. `-m manual` is also default, so this can be simplified to ```python dnsproxy.py -o haproxy```.
+Now just point your DNS Entries to your VPS and the traffic for the needed Domains goes through the Proxy.
 
-It is also possible to specify which proxy list you would like to use, based on country. You can specify that by passing `-c <country>`, where `<country>` is a suffix of any file in the `proxies/` directory. For example, if you wish to generate configuration for a uk based SNI proxy, you can run `python dnsproxy.py -c uk -m sni`. The default country is `us`.
+It is also possible to specify which proxy list you would like to use, based on country. You can specify that by passing `-c <country>`, where `<country>` is a suffix of any file in the `proxies/` directory. For example, if you wish to generate configuration for a uk based SNI proxy, you can run `python dnsproxy.py -c uk -m sni`. The default country is `ch`.
 
-Check the wiki for additional information, along with these links by @trick77:
-- http://trick77.com/2014/03/01/tunlr-style-dns-unblocking-pandora-netflix-hulu-et-al/
-- http://trick77.com/2014/03/02/dns-unblocking-using-dnsmasq-haproxy/
 
 *If you would like to add a service, please send a pull request.*
 
 Output of `dnsproxy.py -h`:
 ```
 usage: dnsproxy.py [-h] [-m {manual,sni,dnat,local}]
-                   [-o {dnsmasq,haproxy,netsh,hosts,rinetd,iptables} [{dnsmasq,haproxy,netsh,hosts,rinetd,iptables} ...]]
-                   [-c COUNTRY] [-d] [--no-test] [--ip IP] [--bind-ip BIND_IP]
-                   [--base-ip BASE_IP] [--base-port BASE_PORT] [--save]
-                   [--output-dir OUTPUT_DIR] [--only [ONLY [ONLY ...]]]
-                   [--skip [SKIP [SKIP ...]]]
+                   [-o {dnsmasq,haproxy,netsh,hosts,rinetd,iptables,iproute2,sniproxy}]
+                   [-c COUNTRY [COUNTRY ...]] [-d] [--ip IP]
+                   [--bind-ip BIND_IP] [--base-ip BASE_IP]
+                   [--base-port BASE_PORT] [--save] [--output-dir OUTPUT_DIR]
+                   [--only [ONLY [ONLY ...]]] [--skip [SKIP [SKIP ...]]]
                    [--dnsmasq-filename DNSMASQ_FILENAME]
                    [--haproxy-filename HAPROXY_FILENAME]
+                   [--sniproxy-filename SNIPROXY_FILENAME]
                    [--iptables-filename IPTABLES_FILENAME]
+                   [--iproute2-filename IPROUTE2_FILENAME]
                    [--netsh-filename NETSH_FILENAME]
                    [--hosts-filename HOSTS_FILENAME]
                    [--rinetd-filename RINETD_FILENAME]
@@ -57,16 +72,14 @@ optional arguments:
   -h, --help            show this help message and exit
   -m {manual,sni,dnat,local}, --mode {manual,sni,dnat,local}
                         Presets for configuration file generation.
-  -o {dnsmasq,haproxy,netsh,hosts,rinetd,iptables} [{dnsmasq,haproxy,netsh,hosts,rinetd,iptables} ...], --output {dnsmasq,haproxy,netsh,hosts,rinetd,iptables} [{dnsmasq,haproxy,netsh,hosts,rinetd,iptables} ...]
+  -o {dnsmasq,haproxy,netsh,hosts,rinetd,iptables,iproute2,sniproxy}, --output {dnsmasq,haproxy,netsh,hosts,rinetd,iptables,iproute2,sniproxy}
                         Which configuration file(s) to generate. This is
                         ignored when not in manual mode.
-  -c COUNTRY, --country COUNTRY
-                        The country to use for generating the configuration.
+  -c COUNTRY [COUNTRY ...], --country COUNTRY [COUNTRY ...]
+                        The country/-ies to use for generating the
+                        configuration (space-separated, e.g. -c us uk).
   -d, --dnat            Specify to use DNAT instead of SNI (Advanced). This is
                         ignored when not in manual mode.
-  --no-test             Specify to skip generating test configuration. This
-                        means that you will not be able to test your setup
-                        with the setup tester.
   --ip IP               Specify the public IP to use
   --bind-ip BIND_IP     Specify the IP that haproxy should bind to
   --base-ip BASE_IP     Specify the base IP from which DNAT should start
@@ -86,8 +99,12 @@ optional arguments:
                         Specify the DNS configuration file name
   --haproxy-filename HAPROXY_FILENAME
                         Specify the haproxy configuration file name
+  --sniproxy-filename SNIPROXY_FILENAME
+                        Specify the sniproxy configuration file name
   --iptables-filename IPTABLES_FILENAME
                         Specify the iptables configuration file name
+  --iproute2-filename IPROUTE2_FILENAME
+                        Specify the iproute2 configuration file name
   --netsh-filename NETSH_FILENAME
                         Specify the netsh configuration file name
   --hosts-filename HOSTS_FILENAME

config-sample.json

@@ -8,7 +8,7 @@
   "base_port": 27200, // Only required for mode "dnat"
   // "local_subnet": 24, // Only required for mode "dnat"
   // "local_device": "eth1", // Only required for mode "dnat"
-  "server_options": "check inter 10s fastinter 2s downinter 2s fall 1800", // Don't touch unless you know what you're doing
+  "server_options": "check init-addr none resolvers mydns", // Don't touch unless you know what you're doing
   "stats": {
     "enabled": true, // Set to true to enable HAProxy stats web page
     "port": 27199,

config.json

@@ -0,0 +1,18 @@
+{
+  "iptables_location": "iptables", // Required to generate iptables rules in mode "dnat"
+  "bind_ip": "*", // The IP address to bind to
+  "public_ip": "", // IP address of your remote server, required for every mode
+  "base_ip": "",
+  // "base_ip": "x.x.x.x", // Only required for modes "dnat" and "local"
+  // "base_ip": "127.0.0.51", // For local
+  "base_port": 27200, // Only required for mode "dnat"
+  // "local_subnet": 24, // Only required for mode "dnat"
+  // "local_device": "eth1", // Only required for mode "dnat"
+  "server_options": "check init-addr none resolvers mydns", // Don't touch unless you know what you're doing
+  "stats": {
+    "enabled": true, // Set to true to enable HAProxy stats web page
+    "port": 27199,
+    "user": "haproxy",
+    "password": "123456"
+  }
+}

dnsproxy.py

@@ -127,7 +127,7 @@ def main(args):
 
     # Choose from the available modes
     if args.mode == "sni":
-        files = ["haproxy", "dnsmasq", "hosts"]
+        files = ["haproxy", "dnsmasq", "hosts", "sniproxy"]
         dnat = False
     elif args.mode == "dnat":
         files = ["haproxy", "dnsmasq", "hosts", "iptables", "iproute2"]
@@ -156,7 +156,13 @@ def main(args):
         print_ips(config)
 
     for output in set(files):
-        if output == "haproxy":
+        if output == "sniproxy":
+            print_firewall(config, dnat=dnat)
+            print ""
+            sniproxy_content = generators.generate_sniproxy(config, dnat=dnat)
+            util.put_contents(args.sniproxy_filename, sniproxy_content, base_dir=args.output_dir)
+            print 'File generated: ' + args.sniproxy_filename
+        elif output == "haproxy":
             print_firewall(config, dnat=dnat)
             if config["stats"]["enabled"] and not config["stats"]["password"]:
                 print ""
@@ -212,8 +218,8 @@ def main(args):
     parser = argparse.ArgumentParser(description="Generate configuration files to setup a tunlr style smart DNS")
 
     parser.add_argument("-m", "--mode", choices=["manual", "sni", "dnat", "local"], default="manual", type=str, help="Presets for configuration file generation.")
-    parser.add_argument("-o", "--output", choices=["dnsmasq", "haproxy", "netsh", "hosts", "rinetd", "iptables", "iproute2"], default=["haproxy"], action="append", help="Which configuration file(s) to generate. This is ignored when not in manual mode.")
-    parser.add_argument("-c", "--country", default="us", type=str, nargs="+", help="The country/-ies to use for generating the configuration (space-separated, e.g. -c us uk).")
+    parser.add_argument("-o", "--output", choices=["dnsmasq", "haproxy", "netsh", "hosts", "rinetd", "iptables", "iproute2", "sniproxy"], default=["sniproxy"], action="append", help="Which configuration file(s) to generate. This is ignored when not in manual mode.")
+    parser.add_argument("-c", "--country", default="ch", type=str, nargs="+", help="The country/-ies to use for generating the configuration (space-separated, e.g. -c us uk).")
     parser.add_argument("-d", "--dnat", action="store_true", help="Specify to use DNAT instead of SNI (Advanced). This is ignored when not in manual mode.")
 
     parser.add_argument("--ip", type=str, default=None, help="Specify the public IP to use")
@@ -229,6 +235,7 @@ def main(args):
 
     parser.add_argument("--dnsmasq-filename", type=str, default="dnsmasq-haproxy.conf", help="Specify the DNS configuration file name")
     parser.add_argument("--haproxy-filename", type=str, default="haproxy.conf", help="Specify the haproxy configuration file name")
+    parser.add_argument("--sniproxy-filename", type=str, default="sniproxy.conf", help="Specify the sniproxy configuration file name")
     parser.add_argument("--iptables-filename", type=str, default="iptables-haproxy.sh", help="Specify the iptables configuration file name")
     parser.add_argument("--iproute2-filename", type=str, default="iproute2-haproxy.sh", help="Specify the iproute2 configuration file name")
     parser.add_argument("--netsh-filename", type=str, default="netsh-haproxy.cmd", help="Specify the netsh configuration file name")

generators/__init__.py

@@ -1,5 +1,6 @@
 
 from haproxy import generate as generate_haproxy
+from sniproxy import generate as generate_sniproxy
 from dnsmasq import generate as generate_dnsmasq
 from iptables import generate as generate_iptables
 from iproute2 import generate as generate_iproute2

generators/dnsmasq.py

@@ -14,10 +14,13 @@ def generate(config, dnat=False):
 
         for chunk in c:
             if not dnat:
-                dnsmasq_content += generate_dns(chunk, public_ip)
+
+                dnsmasq_contentwld = generate_dns(chunk, public_ip)
+                dnsmasq_content += dnsmasq_contentwld.replace("*", "")
+
             else:
-                dnsmasq_content += generate_dns(chunk, current_ip)
-
+                dnsmasq_contentwld += generate_dns(chunk, current_ip)
+                dnsmasq_content = dnsmasq_contentwld.replace("*", "")
     if dnat:
         for group in config["groups"].values():
             for proxy in group["proxies"]:

generators/haproxy.py

@@ -9,8 +9,8 @@ def generate(config, dnat=False):
         current_port = config["base_port"]
     elif dnat:
         return
-
-    haproxy_content = generate_global()
+    haproxy_content = generate_mydns()
+    haproxy_content += generate_global()
     haproxy_content += generate_defaults()
 
     if not dnat:
@@ -85,6 +85,14 @@ def generate_backend_catchall_entry(domain, mode, port, server_options, override
     return result
 
 
+def generate_mydns():
+    result = fmt('resolvers mydns', indent=None)
+    result += fmt('nameserver dns1 8.8.8.8:53')
+    result += fmt('nameserver dns2 8.8.4.4:53')
+    result += os.linesep
+    return result
+
+
 def generate_global():
     result = fmt('global', indent=None)
     result += fmt('daemon')

generators/sniproxy.py

@@ -0,0 +1,103 @@
+from util import fmt, port, chunks
+import os
+
+def generate_startconfig01():
+    result = fmt('# sniproxy example configuration file', indent=None)
+    result += fmt('# lines that start with # are comments', indent=None)
+    result += fmt('# lines with only white space are ignored', indent=None)
+    result += fmt('', indent=None)
+    result += fmt('user daemon', indent=None)
+    result += fmt('', indent=None)    
+    result += fmt('# PID file', indent=None)
+    result += fmt('pidfile /var/run/sniproxy.pid', indent=None)  
+    result += os.linesep
+    return result
+
+def generate_mydns():
+    result = fmt('resolver {', indent=None)
+    result += fmt('nameserver 8.8.8.8')
+    result += fmt('nameserver 8.8.4.4')
+    result += fmt('mode ipv4_only')
+    result += fmt('}', indent=None)
+    result += os.linesep
+    return result
+
+def generate_error():
+    result = fmt('error_log {', indent=None)
+    result += fmt('# Log to the daemon syslog facility')
+    result += fmt('syslog daemon')
+    result += fmt('', indent=None)
+    result += fmt('# Alternatively we could log to file')
+    result += fmt('#filename /var/log/sniproxy/sniproxy.log')
+    result += fmt('', indent=None)    
+    result += fmt('# Control the verbosity of the log')
+    result += fmt('priority notice')
+    result += fmt('}', indent=None)     
+    result += os.linesep
+    return result
+
+def generate_listenhttp():
+    result = fmt('# blocks are delimited with {...}', indent=None)
+    result += fmt('listen 80 {', indent=None)
+    result += fmt('proto http')
+    result += fmt('table hosts')
+    result += fmt('# Fallback backend server to use if we can not parse the client request')
+    result += fmt('fallback localhost:8080')
+    result += fmt('', indent=None)    
+    result += fmt('access_log {')
+    result += fmt('filename /var/log/sniproxy/http_access.log')
+    result += fmt('priority notice')
+    result += fmt('}')
+    result += fmt('}', indent=None)    
+    result += os.linesep
+    return result
+
+def generate_listentls():
+    result = fmt('# blocks are delimited with {...}', indent=None)
+    result += fmt('listen 443 {', indent=None)
+    result += fmt('proto tls')
+    result += fmt('table hosts')
+    result += fmt('', indent=None)    
+    result += fmt('access_log {')
+    result += fmt('filename /var/log/sniproxy/https_access.log')
+    result += fmt('priority notice')
+    result += fmt('}')
+    result += fmt('}', indent=None)    
+    result += os.linesep
+    return result 
+
+
+def generate_hosts01():
+    result = fmt('table hosts{', indent=None)  
+    return result 
+
+def generate_hosts02(): 
+    result = fmt('#    .* *')
+    result += fmt('}', indent=None)    
+    result += os.linesep
+    return result 
+
+def generate(config, dnat=False):
+    sniproxy_content = ""
+    sniproxy_content += generate_startconfig01()
+    sniproxy_content += generate_mydns()
+    sniproxy_content += generate_error()
+    sniproxy_content += generate_listenhttp()
+    sniproxy_content += generate_listentls()
+    sniproxy_content += generate_hosts01()    
+
+    for group in config["groups"].values():
+        for proxy in group["proxies"]:
+            proxy["domain2"] = proxy["domain"].replace('.','\.')
+            if (proxy["domain"].startswith('*')):
+                sniproxy_content += '    .' + proxy["domain2"] + '$ ' +  '*' + '\n'
+            else:
+                sniproxy_content += '    ^' + proxy["domain2"] + ' ' +  proxy["domain"] + '\n'
+
+
+
+
+    sniproxy_content += generate_hosts02()  
+
+
+    return sniproxy_content