diff --git a/README.md b/README.md index 371459b..9e983e7 100644 --- a/README.md +++ b/README.md @@ -19,18 +19,42 @@ or use these lists for other applications like selektor. So we make two files that are structured in YAML: ``` /etc/tor/yaml/torrc-goodnodes.yaml + +--- GoodNodes: + EntryNodes: [] Relays: - IntroductionPoints: - - NODEFINGERPRINT - ... + # ExitNodes will be overwritten by this program + ExitNodes: [] + IntroductionPoints: [] + # use the Onions section to list onion services you want the + # Introduction Points whitelisted - these points may change daily + # Look in tor's notice.log for 'Every introduction point for service' + Onions: [] + # use the Services list to list elays you want the whitelisted + # Look in tor's notice.log for 'Wanted to contact directory mirror' + Services: [] + + By default all sections of the goodnodes.yaml are used as a whitelist. +Use the GoodNodes/Onions list to list onion services you want the +Introduction Points whitelisted - these points may change daily +Look in tor's notice.log for warnings of 'Every introduction point for service' + /etc/tor/yaml/torrc-badnodes.yaml + BadNodes: - ExcludeExitNodes: - BadExit: - # $0000000000000000000000000000000000000007 + # list the internet domains you know are bad so you don't + # waste time trying to download contacts from them. + ExcludeDomains: [] + ExcludeNodes: + # BadExit will be overwritten by this program + BadExit: [] + # list MyBadExit in --bad_sections if you want it used, to exclude nodes + # or any others as a list separated by comma(,) + MyBadExit: [] + ``` That part requires [PyYAML](https://pyyaml.org/wiki/PyYAML) https://github.com/yaml/pyyaml/ or ```ruamel```: do @@ -39,7 +63,7 @@ the advantage of the former is that it preserves comments. (You may have to run this as the Tor user to get RW access to /run/tor/control, in which case the directory for the YAML files must -be group Tor writeable, and its parents group Tor RX.) +be group Tor writeable, and its parent's directories group Tor RX.) Because you don't want to exclude the introduction points to any onion you want to connect to, ```--white_onions``` should whitelist the @@ -47,6 +71,13 @@ introduction points to a comma sep list of onions; we fixed stem to do this: * https://github.com/torproject/stem/issues/96 * https://gitlab.torproject.org/legacy/trac/-/issues/25417 +Use the GoodNodes/Onions list in goodnodes.yaml to list onion services +you want the Introduction Points whitelisted - these points may change daily. +Look in tor's notice.log for 'Every introduction point for service' + +```notice_log``` will parse the notice log for warnings about relays and +services that will then be whitelisted. + ```--torrc_output``` will write the torrc ExcludeNodes configuration to a file. ```--good_contacts``` will write the contact info as a ciiss dictionary @@ -71,7 +102,7 @@ list of fingerprints to ```ExitNodes```, a whitelist of relays to use as exits. 3. clean relays that don't have "good' contactinfo. (implies 1) ```=Empty,NoEmail,NotGood``` -The default is ```=Empty,NotGood``` ; ```NoEmail``` is inherently imperfect +The default is ```Empty,NoEmail,NotGood``` ; ```NoEmail``` is inherently imperfect in that many of the contact-as-an-email are obfuscated, but we try anyway. To be "good" the ContactInfo must: @@ -80,81 +111,9 @@ To be "good" the ContactInfo must: 3. must support getting the file with a valid SSL cert from a recognized authority 4. (not in the spec but added by Python) must use a TLS SSL > v1 5. must have a fingerprint list in the file -6. must have the FP that got us the contactinfo in the fingerprint list in the file, +6. must have the FP that got us the contactinfo in the fingerprint list in the file. + For usage, do ```python3 exclude_badExits.py --help` - -## Usage -``` - -usage: exclude_badExits.py [-h] [--https_cafile HTTPS_CAFILE] - [--proxy_host PROXY_HOST] [--proxy_port PROXY_PORT] - [--proxy_ctl PROXY_CTL] [--torrc TORRC] - [--timeout TIMEOUT] [--good_nodes GOOD_NODES] - [--bad_nodes BAD_NODES] [--bad_on BAD_ON] - [--bad_contacts BAD_CONTACTS] - [--strict_nodes {0,1}] [--wait_boot WAIT_BOOT] - [--points_timeout POINTS_TIMEOUT] - [--log_level LOG_LEVEL] - [--bad_sections BAD_SECTIONS] - [--white_onions WHITE_ONIONS] - [--torrc_output TORRC_OUTPUT] - [--relays_output RELAYS_OUTPUT] - [--good_contacts GOOD_CONTACTS] - -optional arguments: - -h, --help show this help message and exit - --https_cafile HTTPS_CAFILE - Certificate Authority file (in PEM) - --proxy_host PROXY_HOST, --proxy-host PROXY_HOST - proxy host - --proxy_port PROXY_PORT, --proxy-port PROXY_PORT - proxy control port - --proxy_ctl PROXY_CTL, --proxy-ctl PROXY_CTL - control socket - or port - --torrc TORRC torrc to check for suggestions - --timeout TIMEOUT proxy download connect timeout - --good_nodes GOOD_NODES - Yaml file of good info that should not be excluded - --bad_nodes BAD_NODES - Yaml file of bad nodes that should also be excluded - --bad_on BAD_ON comma sep list of conditions - Empty,NoEmail,NotGood - --bad_contacts BAD_CONTACTS - Yaml file of bad contacts that bad FPs are using - --strict_nodes {0,1} Set StrictNodes: 1 is less anonymous but more secure, - although some sites may be unreachable - --wait_boot WAIT_BOOT - Seconds to wait for Tor to booststrap - --points_timeout POINTS_TIMEOUT - Timeout for getting introduction points - must be long - >120sec. 0 means disabled looking for IPs - --log_level LOG_LEVEL - 10=debug 20=info 30=warn 40=error - --bad_sections BAD_SECTIONS - sections of the badnodes.yaml to use, comma separated, - '' BROKEN - --white_onions WHITE_ONIONS - comma sep. list of onions to whitelist their - introduction points - BROKEN - --torrc_output TORRC_OUTPUT - Write the torrc configuration to a file - --relays_output RELAYS_OUTPUT - Write the download relays in json to a file - --good_contacts GOOD_CONTACTS - Write the proof data of the included nodes to a YAML - file - -This extends nusenu's basic idea of using the stem library to dynamically -exclude nodes that are likely to be bad by putting them on the ExcludeNodes or -ExcludeExitNodes setting of a running Tor. * -https://github.com/nusenu/noContactInfo_Exit_Excluder * -https://github.com/TheSmashy/TorExitRelayExclude The basic idea is to exclude -Exit nodes that do not have ContactInfo: * -https://github.com/nusenu/ContactInfo-Information-Sharing-Specification That -can be extended to relays that do not have an email in the contact, or to -relays that do not have ContactInfo that is verified to include them. - -``` - diff --git a/exclude_badExits.bash b/exclude_badExits.bash index 6120fc9..e08ba05 100644 --- a/exclude_badExits.bash +++ b/exclude_badExits.bash @@ -3,25 +3,37 @@ PROG=exclude_badExits.py SOCKS_PORT=9050 +SOCKS_HOST=127.0.0.1 CAFILE=/etc/ssl/certs/ca-certificates.crt # you may have a special python for installed packages EXE=`which python3.bash` -$EXE exclude_badExits.py --help > exclude_badExits.hlp & +$EXE exclude_badExits.py --help > exclude_badExits.txt & +$EXE -c 'from exclude_badExits import __doc__; print(__doc__)' >exclude_badExits.md # an example of running exclude_badExits with full debugging -# expected to take an hour or so +# expected to 20 minutes or so declare -a LARGS LARGS=( + # --saved_only # --strict_nodes 1 - --points_timeout 120 + --points_timeout 150 --log_level 10 --https_cafile $CAFILE - ) +) +[ -z "$socks_proxy" ] || \ LARGS+=( - --proxy-host 127.0.0.1 + --proxy-host $SOCKS_HOST --proxy-port $SOCKS_PORT ) +if [ -f /var/lib/tor/.SelekTOR/3xx/cache/9050/notice.log ] ; then + LARGS+=(--notice_log /var/lib/tor/.SelekTOR/3xx/cache/9050/notice.log) +fi + +if [ -d /var/lib/tor/hs ] ; then + LARGS+=( --hs_dir /var/lib/tor/hs ) +fi + if [ -f '/run/tor/control' ] ; then LARGS+=(--proxy-ctl '/run/tor/control' ) else diff --git a/exclude_badExits.py b/exclude_badExits.py index 6e4fc5e..7d62086 100644 --- a/exclude_badExits.py +++ b/exclude_badExits.py @@ -17,7 +17,37 @@ or to relays that do not have ContactInfo that is verified to include them. """ __prolog__ = __doc__ -__doc__ +="""But there's a problem, and your Tor notice.log will tell you about it: +sGOOD_NODES = """ +--- +GoodNodes: + EntryNodes: [] + Relays: + # ExitNodes will be overwritten by this program + ExitNodes: [] + IntroductionPoints: [] + # use the Onions section to list onion services you want the + # Introduction Points whitelisted - these points may change daily + # Look in tor's notice.log for 'Every introduction point for service' + Onions: [] + # use the Services list to list elays you want the whitelisted + # Look in tor's notice.log for 'Wanted to contact directory mirror' + Services: [] +""" + +sBAD_NODES = """ +BadNodes: + # list the internet domains you know are bad so you don't + # waste time trying to download contacts from them. + ExcludeDomains: [] + ExcludeNodes: + # BadExit will be overwritten by this program + BadExit: [] + # list MyBadExit in --bad_sections if you want it used, to exclude nodes + # or any others as a list separated by comma(,) + MyBadExit: [] +""" + +__doc__ +=f"""But there's a problem, and your Tor notice.log will tell you about it: you could exclude the relays needed to access hidden services or mirror directories. So we need to add to the process the concept of a whitelist. In addition, we may have our own blacklist of nodes we want to exclude, @@ -26,18 +56,16 @@ or use these lists for other applications like selektor. So we make two files that are structured in YAML: ``` /etc/tor/yaml/torrc-goodnodes.yaml -GoodNodes: - Relays: - IntroductionPoints: - - NODEFINGERPRINT - ... +{sGOOD_NODES} + By default all sections of the goodnodes.yaml are used as a whitelist. +Use the GoodNodes/Onions list to list onion services you want the +Introduction Points whitelisted - these points may change daily +Look in tor's notice.log for warnings of 'Every introduction point for service' + /etc/tor/yaml/torrc-badnodes.yaml -BadNodes: - ExcludeExitNodes: - BadExit: - - 0000000000000000000000000000000000000007 +{sBAD_NODES} ``` That part requires [PyYAML](https://pyyaml.org/wiki/PyYAML) https://github.com/yaml/pyyaml/ or ```ruamel```: do @@ -46,7 +74,7 @@ the advantage of the former is that it preserves comments. (You may have to run this as the Tor user to get RW access to /run/tor/control, in which case the directory for the YAML files must -be group Tor writeable, and its parents group Tor RX.) +be group Tor writeable, and its parent's directories group Tor RX.) Because you don't want to exclude the introduction points to any onion you want to connect to, ```--white_onions``` should whitelist the @@ -54,6 +82,13 @@ introduction points to a comma sep list of onions; we fixed stem to do this: * https://github.com/torproject/stem/issues/96 * https://gitlab.torproject.org/legacy/trac/-/issues/25417 +Use the GoodNodes/Onions list in goodnodes.yaml to list onion services +you want the Introduction Points whitelisted - these points may change daily. +Look in tor's notice.log for 'Every introduction point for service' + +```notice_log``` will parse the notice log for warnings about relays and +services that will then be whitelisted. + ```--torrc_output``` will write the torrc ExcludeNodes configuration to a file. ```--good_contacts``` will write the contact info as a ciiss dictionary @@ -78,7 +113,7 @@ list of fingerprints to ```ExitNodes```, a whitelist of relays to use as exits. 3. clean relays that don't have "good' contactinfo. (implies 1) ```=Empty,NoEmail,NotGood``` -The default is ```=Empty,NotGood``` ; ```NoEmail``` is inherently imperfect +The default is ```Empty,NoEmail,NotGood``` ; ```NoEmail``` is inherently imperfect in that many of the contact-as-an-email are obfuscated, but we try anyway. To be "good" the ContactInfo must: @@ -87,7 +122,8 @@ To be "good" the ContactInfo must: 3. must support getting the file with a valid SSL cert from a recognized authority 4. (not in the spec but added by Python) must use a TLS SSL > v1 5. must have a fingerprint list in the file -6. must have the FP that got us the contactinfo in the fingerprint list in the file, +6. must have the FP that got us the contactinfo in the fingerprint list in the file. + For usage, do ```python3 exclude_badExits.py --help` @@ -175,28 +211,12 @@ sEXCLUDE_EXIT_GROUP = 'ExcludeNodes' sINCLUDE_EXIT_KEY = 'ExitNodes' oBAD_ROOT = 'BadNodes' -aBAD_NODES = safe_load(""" -BadNodes: - ExcludeDomains: [] - ExcludeNodes: - # BadExit will be overwritten - BadExit: [] - # list MyBadExit in --bad_sections if you want it used - MyBadExit: [] -""") +aBAD_NODES = safe_load(sBAD_NODES) sGOOD_ROOT = 'GoodNodes' sINCLUDE_GUARD_KEY = 'EntryNodes' sEXCLUDE_DOMAINS = 'ExcludeDomains' -oGOOD_NODES = safe_load(""" -GoodNodes: - EntryNodes: [] - Relays: - ExitNodes: [] - IntroductionPoints: [] - Onions: [] - Services: [] -""") +aGOOD_NODES = safe_load(sGOOD_NODES) lKNOWN_NODNS = [] tMAYBE_NODNS = set() @@ -230,13 +250,13 @@ def lYamlBadNodes(sFile, return l def lYamlGoodNodes(sFile='/etc/tor/torrc-goodnodes.yaml'): - global oGOOD_NODES + global aGOOD_NODES l = [] if not yaml: return l if os.path.exists(sFile): with open(sFile, 'rt') as oFd: o = safe_load(oFd) - oGOOD_NODES = o + aGOOD_NODES = o if 'EntryNodes' in o[sGOOD_ROOT].keys(): l = o[sGOOD_ROOT]['EntryNodes'] # yq '.Nodes.IntroductionPoints|.[]' < /etc/tor/torrc-goodnodes.yaml @@ -644,9 +664,9 @@ def oMainArgparser(_=None): default='127.0.0.1', help='proxy host') parser.add_argument('--proxy_port', '--proxy-port', default=9050, type=int, - help='proxy control port') + help='proxy socks port') parser.add_argument('--proxy_ctl', '--proxy-ctl', - default='/run/tor/control' if os.path.exists('/run/tor/control') else 9051, + default='/run/tor/control' if os.path.exists('/run/tor/control') else '9051', type=str, help='control socket - or port') @@ -689,9 +709,12 @@ def oMainArgparser(_=None): parser.add_argument('--torrc_output', type=str, default=os.path.join(ETC_DIR, 'torrc.new'), help="Write the torrc configuration to a file") + parser.add_argument('--hs_dir', type=str, + default='/var/lib/tor', + help="Parse the files name hostname below this dir to find Hidden Services to whitelist") parser.add_argument('--notice_log', type=str, default='', - help="Parse the notice log for relays and services (not yet)") + help="Parse the notice log for relays and services") parser.add_argument('--relays_output', type=str, default=os.path.join(ETC_DIR, 'relays.json'), help="Write the download relays in json to a file") @@ -718,23 +741,23 @@ def vwrite_good_contacts(oargs): yaml.dump(aBAD_CONTACTS_DB, oFYaml) oFYaml.close() -def vwrite_badnodes(oargs, aBAD_NODES, slen): +def vwrite_badnodes(oargs, aBAD_NODES, slen, stag): if not aBAD_NODES: return tmp = oargs.bad_nodes +'.tmp' bak = oargs.bad_nodes +'.bak' with open(tmp, 'wt') as oFYaml: yaml.dump(aBAD_NODES, oFYaml) - LOG.info(f"Wrote {slen} to {oargs.bad_nodes}") + LOG.info(f"Wrote {slen} to {stag} in {oargs.bad_nodes}") oFYaml.close() if os.path.exists(oargs.bad_nodes): os.rename(oargs.bad_nodes, bak) os.rename(tmp, oargs.bad_nodes) -def vwrite_goodnodes(oargs, oGOOD_NODES, ilen): +def vwrite_goodnodes(oargs, aGOOD_NODES, ilen): tmp = oargs.good_nodes +'.tmp' bak = oargs.good_nodes +'.bak' with open(tmp, 'wt') as oFYaml: - yaml.dump(oGOOD_NODES, oFYaml) + yaml.dump(aGOOD_NODES, oFYaml) LOG.info(f"Wrote {ilen} good relays to {oargs.good_nodes}") oFYaml.close() if os.path.exists(oargs.good_nodes): @@ -1022,23 +1045,31 @@ def tWhitelistSet(oargs, controller): LOG.info(f"lYamlGoodNodes {len(twhitelist_set)} EntryNodes from {oargs.good_nodes}") t = set() - if sGOOD_ROOT in oGOOD_NODES and 'Relays' in oGOOD_NODES[sGOOD_ROOT] and \ - 'IntroductionPoints' in oGOOD_NODES[sGOOD_ROOT]['Relays'].keys(): - t = set(oGOOD_NODES[sGOOD_ROOT]['Relays']['IntroductionPoints']) + if 'IntroductionPoints' in aGOOD_NODES[sGOOD_ROOT]['Relays'].keys(): + t = set(aGOOD_NODES[sGOOD_ROOT]['Relays']['IntroductionPoints']) + + if oargs.hs_dir and os.path.exists(oargs.hs_dir): + for (dirpath, dirnames, filenames,) in os.walk(oargs.hs_dir): + for f in filenames: + if f != 'hostname': continue + with open(os.path.join(dirpath, f), 'rt') as oFd: + son = oFd.read() + t.update(son) + LOG.info(f"Added {son} to the list for Introduction Points") if oargs.notice_log and os.path.exists(oargs.notice_log): tmp = tempfile.mktemp() i = os.system(f"grep 'Every introduction point for service' {oargs.notice_log} |sed -e 's/.* service //' -e 's/ is .*//'|sort -u |sed -e '/ /d' > {tmp}") if i: with open(tmp, 'rt') as oFd: - lnew = oFd.readlines() - t.update(set(lnew)) + tnew = {elt.strip() for elt in oFd.readlines()} + t.update(tnew) LOG.info(f"Whitelist {len(lnew)} services from {oargs.notice_log}") os.remove(tmp) w = set() - if sGOOD_ROOT in oGOOD_NODES and 'Services' in oGOOD_NODES[sGOOD_ROOT].keys(): - w = set(oGOOD_NODES[sGOOD_ROOT]['Services']) + if sGOOD_ROOT in aGOOD_NODES and 'Services' in aGOOD_NODES[sGOOD_ROOT].keys(): + w = set(aGOOD_NODES[sGOOD_ROOT]['Services']) if len(w) > 0: LOG.info(f"Whitelist {len(w)} relays from {sGOOD_ROOT}/Services") @@ -1054,10 +1085,10 @@ def tWhitelistSet(oargs, controller): twhitelist_set.update(w) w = set() - if 'Onions' in oGOOD_NODES[sGOOD_ROOT].keys(): + if 'Onions' in aGOOD_NODES[sGOOD_ROOT].keys(): # Provides the descriptor for a hidden service. The **address** is the # '.onion' address of the hidden service - w = set(oGOOD_NODES[sGOOD_ROOT]['Onions']) + w = set(aGOOD_NODES[sGOOD_ROOT]['Onions']) if oargs.white_onions: w.update(oargs.white_onions.split(',')) if oargs.points_timeout > 0: @@ -1088,7 +1119,7 @@ def iMain(lArgs): global aGOOD_CONTACTS_FPS global aBAD_CONTACTS_DB global aBAD_NODES - global oGOOD_NODES + global aGOOD_NODES global lKNOWN_NODNS global aRELAYS_DB global aRELAYS_DB_INDEX @@ -1198,7 +1229,7 @@ def iMain(lArgs): with open(oargs.torrc_output, 'wt') as oFTorrc: oFTorrc.write(f"{sEXCLUDE_EXIT_GROUP} {','.join(texclude_set)}\n") oFTorrc.write(f"{sINCLUDE_EXIT_KEY} {','.join(aGOOD_CONTACTS_FPS.keys())}\n") - oFTorrc.write(f"{sINCLUDE_GUARD_KEY} {','.join(oGOOD_NODES[sGOOD_ROOT]['EntryNodes'])}\n") + oFTorrc.write(f"{sINCLUDE_GUARD_KEY} {','.join(aGOOD_NODES[sGOOD_ROOT]['EntryNodes'])}\n") LOG.info(f"Wrote tor configuration to {oargs.torrc_output}") oFTorrc.close() @@ -1214,12 +1245,13 @@ def iMain(lArgs): aBAD_NODES[oBAD_ROOT][sEXCLUDE_EXIT_GROUP]['BadExit'] = list(texclude_set) aBAD_NODES[oBAD_ROOT][sEXCLUDE_DOMAINS] = lKNOWN_NODNS if oargs.bad_nodes: - vwrite_badnodes(oargs, aBAD_NODES, str(len(texclude_set))) + stag = sEXCLUDE_EXIT_GROUP + '/BadExit' + vwrite_badnodes(oargs, aBAD_NODES, str(len(texclude_set)), stag) - oGOOD_NODES['GoodNodes']['Relays']['ExitNodes'] = list(aGOOD_CONTACTS_FPS.keys()) + aGOOD_NODES['GoodNodes']['Relays']['ExitNodes'] = list(aGOOD_CONTACTS_FPS.keys()) # EntryNodes are readony if oargs.good_nodes: - vwrite_goodnodes(oargs, oGOOD_NODES, len(aGOOD_CONTACTS_FPS.keys())) + vwrite_goodnodes(oargs, aGOOD_NODES, len(aGOOD_CONTACTS_FPS.keys())) vwritefinale(oargs) @@ -1245,15 +1277,15 @@ def iMain(lArgs): LOG.debug(repr(l)) retval += 1 - if 'EntryNodes' in oGOOD_NODES[sGOOD_ROOT].keys(): + if 'EntryNodes' in aGOOD_NODES[sGOOD_ROOT].keys(): try: - LOG.info(f"{sINCLUDE_GUARD_KEY} {len(oGOOD_NODES[sGOOD_ROOT]['EntryNodes'])} guard nodes") + LOG.info(f"{sINCLUDE_GUARD_KEY} {len(aGOOD_NODES[sGOOD_ROOT]['EntryNodes'])} guard nodes") # FixMe for now override StrictNodes it may be unusable otherwise controller.set_conf(sINCLUDE_GUARD_KEY, - oGOOD_NODES[sGOOD_ROOT]['EntryNodes']) + aGOOD_NODES[sGOOD_ROOT]['EntryNodes']) except (Exception, stem.InvalidRequest, stem.SocketClosed,) as e: # noqa LOG.error(f"Failed setting {sINCLUDE_GUARD_KEY} guard nodes in Tor {e}") - LOG.debug(repr(list(oGOOD_NODES[sGOOD_ROOT]['EntryNodes']))) + LOG.debug(repr(list(aGOOD_NODES[sGOOD_ROOT]['EntryNodes']))) retval += 1 cur = controller.get_conf('StrictNodes') diff --git a/exclude_badExits.txt b/exclude_badExits.txt new file mode 100644 index 0000000..8e0b180 --- /dev/null +++ b/exclude_badExits.txt @@ -0,0 +1,76 @@ +usage: exclude_badExits.py [-h] [--https_cafile HTTPS_CAFILE] + [--proxy_host PROXY_HOST] [--proxy_port PROXY_PORT] + [--proxy_ctl PROXY_CTL] [--torrc TORRC] + [--timeout TIMEOUT] [--good_nodes GOOD_NODES] + [--bad_nodes BAD_NODES] [--bad_on BAD_ON] + [--bad_contacts BAD_CONTACTS] [--saved_only] + [--strict_nodes {0,1}] [--wait_boot WAIT_BOOT] + [--points_timeout POINTS_TIMEOUT] + [--log_level LOG_LEVEL] + [--bad_sections BAD_SECTIONS] + [--white_onions WHITE_ONIONS] + [--torrc_output TORRC_OUTPUT] [--hs_dir HS_DIR] + [--notice_log NOTICE_LOG] + [--relays_output RELAYS_OUTPUT] + [--wellknown_output WELLKNOWN_OUTPUT] + [--good_contacts GOOD_CONTACTS] + +optional arguments: + -h, --help show this help message and exit + --https_cafile HTTPS_CAFILE + Certificate Authority file (in PEM) + --proxy_host PROXY_HOST, --proxy-host PROXY_HOST + proxy host + --proxy_port PROXY_PORT, --proxy-port PROXY_PORT + proxy control port + --proxy_ctl PROXY_CTL, --proxy-ctl PROXY_CTL + control socket - or port + --torrc TORRC torrc to check for suggestions + --timeout TIMEOUT proxy download connect timeout + --good_nodes GOOD_NODES + Yaml file of good info that should not be excluded + --bad_nodes BAD_NODES + Yaml file of bad nodes that should also be excluded + --bad_on BAD_ON comma sep list of conditions - Empty,NoEmail,NotGood + --bad_contacts BAD_CONTACTS + Yaml file of bad contacts that bad FPs are using + --saved_only Just use the info in the last *.yaml files without + querying the Tor controller + --strict_nodes {0,1} Set StrictNodes: 1 is less anonymous but more secure, + although some onion sites may be unreachable + --wait_boot WAIT_BOOT + Seconds to wait for Tor to booststrap + --points_timeout POINTS_TIMEOUT + Timeout for getting introduction points - must be long + >120sec. 0 means disabled looking for IPs + --log_level LOG_LEVEL + 10=debug 20=info 30=warn 40=error + --bad_sections BAD_SECTIONS + sections of the badnodes.yaml to use, in addition to + BadExit, comma separated + --white_onions WHITE_ONIONS + comma sep. list of onions to whitelist their + introduction points - BROKEN + --torrc_output TORRC_OUTPUT + Write the torrc configuration to a file + --hs_dir HS_DIR Parse the files name hostname below this dir to find + Hidden Services to whitelist + --notice_log NOTICE_LOG + Parse the notice log for relays and services + --relays_output RELAYS_OUTPUT + Write the download relays in json to a file + --wellknown_output WELLKNOWN_OUTPUT + Write the well-known files to a directory + --good_contacts GOOD_CONTACTS + Write the proof data of the included nodes to a YAML + file + +This extends nusenu's basic idea of using the stem library to dynamically +exclude nodes that are likely to be bad by putting them on the ExcludeNodes or +ExcludeExitNodes setting of a running Tor. * +https://github.com/nusenu/noContactInfo_Exit_Excluder * +https://github.com/TheSmashy/TorExitRelayExclude The basic idea is to exclude +Exit nodes that do not have ContactInfo: * +https://github.com/nusenu/ContactInfo-Information-Sharing-Specification That +can be extended to relays that do not have an email in the contact, or to +relays that do not have ContactInfo that is verified to include them. diff --git a/support_onions.py b/support_onions.py index e68edcc..abdc325 100644 --- a/support_onions.py +++ b/support_onions.py @@ -33,44 +33,39 @@ bHAVE_TORR = shutil.which('tor-resolve') # in the wild we'll keep a copy here so we can avoid restesting yKNOWN_NODNS = """ --- - - for-privacy.net - - backup.spekadyon.org - - verification-for-nusenu.net - - prsv.ch - - ezyn.de - - dfri.se - - dtf.contact - - galtland.network - - dotsrc.org - - nicdex.com - - unzane.com - - a9.wtf - - tor.skankhunt42.pw - - tor-exit-3.aa78i2efsewr0neeknk.xyz - - privacysvcs.net - - apt96.com - - mkg20001.io - - kryptonit.org - - sebastian-elisa-pfeifer.eu - - nx42.de - - www.defcon.org - 0x0.is - - transliberation.today - - tor-exit-2.aa78i2efsewr0neeknk.xyz - - interfesse.net + - a9.wtf + - apt96.com - axims.net - - a9.wtf + - backup.spekadyon.org + - dfri.se + - dotsrc.org + - dtf.contact + - ezyn.de + - for-privacy.net + - galtland.network - heraldonion.org + - interfesse.net + - kryptonit.org - linkspartei.org + - mkg20001.io + - nicdex.com + - nx42.de - pineapple.cx - privacylayer.xyz + - privacysvcs.net - prsv.ch + - sebastian-elisa-pfeifer.eu - thingtohide.nl - - tor-exit-2.aa78i2efsewr0neeknk.xyz - - tor-exit-3.aa78i2efsewr0neeknk.xyz + - tor-exit-2.aa78i2efsewr0neeknk.xyz + - tor-exit-3.aa78i2efsewr0neeknk.xyz - tor.dlecan.com + - tor.skankhunt42.pw + - transliberation.today - tuxli.org + - unzane.com - verification-for-nusenu.net + - www.defcon.org """ # - 0x0.is # - aklad5.com @@ -246,7 +241,8 @@ def lIntroductionPoints(controller=None, lOnions=[], itimeout=120, log_level=10) l += lp except (Empty, Timeout,) as e: # noqa LOG.warn(f"Timed out getting introduction points for {elt}") - continue + except stem.DescriptorUnavailable as e: + LOG.error(e) except Exception as e: LOG.exception(e) return l