stem_examples/docs/download_descriptor.html

250 lines
24 KiB
HTML
Raw Normal View History

2024-01-14 15:28:53 +01:00
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Download Tor Descriptors &mdash; Stem 1.8.1-maint documentation</title>
<link rel="stylesheet" href="../../_static/haiku.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../_static/style.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../../',
VERSION: '1.8.1-maint',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../../_static/jquery.js"></script>
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<link rel="shortcut icon" href="../../_static/favicon.png"/>
<link rel="top" title="Stem 1.8.1-maint documentation" href="../../index.html" />
</head>
<body>
<div class="header"><img class="rightlogo" src="../../_static/logo.png" alt="Logo"/><h1 class="heading"><a href="../../index.html">
<span>Stem Docs</span></a></h1>
<h2 class="heading"><span>Download Tor Descriptors</span></h2>
</div>
<div class="topnav">
<p>
<ul id="navbar">
<li><a href="../../index.html">Home</a></li>
<li><a href="../../tutorials.html">Tutorials</a>
<ul>
<li><a href="../the_little_relay_that_could.html">Hello World</a></li>
<li><a href="../to_russia_with_love.html">Client Usage</a></li>
<li><a href="../tortoise_and_the_hare.html">Event Listening</a></li>
<li><a href="../over_the_river.html">Hidden Services</a></li>
<li><a href="../mirror_mirror_on_the_wall.html">Tor Descriptors</a></li>
<li><a href="../east_of_the_sun.html">Utilities</a></li>
<li><a href="../down_the_rabbit_hole.html">Interpreter</a></li>
<li><a href="../double_double_toil_and_trouble.html">Examples</a></li>
</ul>
</li>
<li><a href="../../api.html">API</a>
<ul>
<li><a href="../../api/control.html">stem.control</a></li>
<li><a href="../../api/connection.html">stem.connection</a></li>
<li><a href="../../api/socket.html">stem.socket</a></li>
<li><a href="../../api/process.html">stem.process</a></li>
<li><a href="../../api/response.html">stem.response</a></li>
<li><a href="../../api/exit_policy.html">stem.exit_policy</a></li>
<li><a href="../../api/version.html">stem.version</a></li>
<li><a href="../../api.html#descriptors">Descriptors</a></li>
<li><a href="../../api.html#utilities">Utilities</a></li>
</ul>
</li>
<li><a href="https://trac.torproject.org/projects/tor/wiki/doc/stem">Development</a>
<ul>
<li><a href="../../faq.html">FAQ</a></li>
<li><a href="../../change_log.html">Change Log</a></li>
<li><a href="https://github.com/torproject/stem/issues/">Bug Tracker</a></li>
<li><a href="https://jenkins.torproject.org/job/stem-tor-ci/">Jenkins</a></li>
<li><a href="../../download.html">Download</a></li>
</ul>
</li>
<li><a href="../../faq.html#where-can-i-get-help">Contact</a>
<ul>
<li><a href="https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev">Email List</a></li>
<li><a href="https://www.torproject.org/about/contact.html.en#irc">IRC</a></li>
<li><a href="https://www.atagar.com/contact/">Author</a></li>
</ul>
</li>
</ul>
</p>
</div>
<div class="content">
<div class="section" id="download-tor-descriptors">
<h1>Download Tor Descriptors<a class="headerlink" href="#download-tor-descriptors" title="Permalink to this headline"></a></h1>
<a class="reference external image-reference" href="../double_double_toil_and_trouble.html"><img alt="../../_images/back.png" src="../../_images/back.png" /></a>
<p>Tor relays provide a mirror for the tor relay descriptors it has cached.
These are available from its ORPort using <a class="reference external" href="https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt">Tor's wire protocol</a>, and optionally
with http as well from a <a class="reference external" href="https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt">DirPort</a>.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="sd">&quot;&quot;&quot;</span>
<span class="sd">Simple script to dowload a descriptor from Tor&#39;s ORPort or DirPort.</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="kn">import</span> <span class="nn">collections</span>
<span class="kn">import</span> <span class="nn">getopt</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">stem</span>
<span class="kn">import</span> <span class="nn">stem.descriptor.remote</span>
<span class="kn">import</span> <span class="nn">stem.util.connection</span>
<span class="kn">import</span> <span class="nn">stem.util.tor_tools</span>
<span class="c"># By default downloading moria1&#39;s server descriptor from itself.</span>
<span class="n">DEFAULT_ARGS</span> <span class="o">=</span> <span class="p">{</span>
<span class="s">&#39;descriptor_type&#39;</span><span class="p">:</span> <span class="s">&#39;server&#39;</span><span class="p">,</span>
<span class="s">&#39;fingerprint&#39;</span><span class="p">:</span> <span class="s">&#39;9695DFC35FFEB861329B9F1AB04C46397020CE31&#39;</span><span class="p">,</span>
<span class="s">&#39;download_from&#39;</span><span class="p">:</span> <span class="n">stem</span><span class="o">.</span><span class="n">DirPort</span><span class="p">(</span><span class="s">&#39;128.31.0.34&#39;</span><span class="p">,</span> <span class="mi">9131</span><span class="p">),</span>
<span class="s">&#39;print_help&#39;</span><span class="p">:</span> <span class="bp">False</span><span class="p">,</span>
<span class="p">}</span>
<span class="n">VALID_TYPES</span> <span class="o">=</span> <span class="p">(</span><span class="s">&#39;server&#39;</span><span class="p">,</span> <span class="s">&#39;extrainfo&#39;</span><span class="p">,</span> <span class="s">&#39;consensus&#39;</span><span class="p">)</span>
<span class="n">HELP_TEXT</span> <span class="o">=</span> <span class="s">&quot;&quot;&quot;</span><span class="se">\</span>
<span class="s">Downloads a descriptor through Tor&#39;s ORPort or DirPort.</span>
<span class="s"> -t, --type TYPE descriptor type to download, options are:</span>
<span class="s"> </span><span class="si">%s</span><span class="s"></span>
<span class="s"> -f, --fingerprint FP relay to download the descriptor of</span>
<span class="s"> --orport ADDRESS:PORT ORPort to download from</span>
<span class="s"> --dirport ADDRESS:PORT DirPort to download from</span>
<span class="s"> -h, --help presents this help</span>
<span class="s">&quot;&quot;&quot;</span> <span class="o">%</span> <span class="s">&#39;, &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">VALID_TYPES</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">parse</span><span class="p">(</span><span class="n">argv</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Parses our arguments, providing a named tuple with their values.</span>
<span class="sd"> :param list argv: input arguments to be parsed</span>
<span class="sd"> :returns: a **named tuple** with our parsed arguments</span>
<span class="sd"> :raises: **ValueError** if we got an invalid argument</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="n">args</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">DEFAULT_ARGS</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">recognized_args</span><span class="p">,</span> <span class="n">unrecognized_args</span> <span class="o">=</span> <span class="n">getopt</span><span class="o">.</span><span class="n">getopt</span><span class="p">(</span><span class="n">argv</span><span class="p">,</span> <span class="s">&#39;t:f:h&#39;</span><span class="p">,</span> <span class="p">[</span><span class="s">&#39;type=&#39;</span><span class="p">,</span> <span class="s">&#39;fingerprint=&#39;</span><span class="p">,</span> <span class="s">&#39;orport=&#39;</span><span class="p">,</span> <span class="s">&#39;dirport=&#39;</span><span class="p">,</span> <span class="s">&#39;help&#39;</span><span class="p">])</span>
<span class="k">if</span> <span class="n">unrecognized_args</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">getopt</span><span class="o">.</span><span class="n">GetoptError</span><span class="p">(</span><span class="s">&quot;&#39;</span><span class="si">%s</span><span class="s">&#39; aren&#39;t recognized arguments&quot;</span> <span class="o">%</span> <span class="s">&quot;&#39;, &#39;&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">unrecognized_args</span><span class="p">))</span>
<span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">exc</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">&#39;</span><span class="si">%s</span><span class="s"> (for usage provide --help)&#39;</span> <span class="o">%</span> <span class="n">exc</span><span class="p">)</span>
<span class="k">for</span> <span class="n">opt</span><span class="p">,</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">recognized_args</span><span class="p">:</span>
<span class="k">if</span> <span class="n">opt</span> <span class="ow">in</span> <span class="p">(</span><span class="s">&#39;-t&#39;</span><span class="p">,</span> <span class="s">&#39;--type&#39;</span><span class="p">):</span>
<span class="k">if</span> <span class="n">arg</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">VALID_TYPES</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">&quot;&#39;</span><span class="si">%s</span><span class="s">&#39; isn&#39;t a recognized decriptor type, options are: </span><span class="si">%s</span><span class="s">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="s">&#39;, &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">VALID_TYPES</span><span class="p">)))</span>
<span class="n">args</span><span class="p">[</span><span class="s">&#39;descriptor_type&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">arg</span>
<span class="k">elif</span> <span class="n">opt</span> <span class="ow">in</span> <span class="p">(</span><span class="s">&#39;-f&#39;</span><span class="p">,</span> <span class="s">&#39;--fingerprint&#39;</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">stem</span><span class="o">.</span><span class="n">util</span><span class="o">.</span><span class="n">tor_tools</span><span class="o">.</span><span class="n">is_valid_fingerprint</span><span class="p">(</span><span class="n">arg</span><span class="p">):</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">&quot;&#39;</span><span class="si">%s</span><span class="s">&#39; isn&#39;t a relay fingerprint&quot;</span> <span class="o">%</span> <span class="n">arg</span><span class="p">)</span>
<span class="n">args</span><span class="p">[</span><span class="s">&#39;fingerprint&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">arg</span>
<span class="k">elif</span> <span class="n">opt</span> <span class="ow">in</span> <span class="p">(</span><span class="s">&#39;--orport&#39;</span><span class="p">,</span> <span class="s">&#39;--dirport&#39;</span><span class="p">):</span>
<span class="k">if</span> <span class="s">&#39;:&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">arg</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">&quot;&#39;</span><span class="si">%s</span><span class="s">&#39; should be of the form &#39;address:port&#39;&quot;</span> <span class="o">%</span> <span class="n">arg</span><span class="p">)</span>
<span class="n">address</span><span class="p">,</span> <span class="n">port</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">rsplit</span><span class="p">(</span><span class="s">&#39;:&#39;</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">stem</span><span class="o">.</span><span class="n">util</span><span class="o">.</span><span class="n">connection</span><span class="o">.</span><span class="n">is_valid_ipv4_address</span><span class="p">(</span><span class="n">address</span><span class="p">):</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">&quot;&#39;</span><span class="si">%s</span><span class="s">&#39; isn&#39;t a valid IPv4 address&quot;</span> <span class="o">%</span> <span class="n">address</span><span class="p">)</span>
<span class="k">elif</span> <span class="ow">not</span> <span class="n">stem</span><span class="o">.</span><span class="n">util</span><span class="o">.</span><span class="n">connection</span><span class="o">.</span><span class="n">is_valid_port</span><span class="p">(</span><span class="n">port</span><span class="p">):</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">&quot;&#39;</span><span class="si">%s</span><span class="s">&#39; isn&#39;t a valid port number&quot;</span> <span class="o">%</span> <span class="n">port</span><span class="p">)</span>
<span class="n">endpoint_class</span> <span class="o">=</span> <span class="n">stem</span><span class="o">.</span><span class="n">ORPort</span> <span class="k">if</span> <span class="n">opt</span> <span class="o">==</span> <span class="s">&#39;--orport&#39;</span> <span class="k">else</span> <span class="n">stem</span><span class="o">.</span><span class="n">DirPort</span>
<span class="n">args</span><span class="p">[</span><span class="s">&#39;download_from&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">endpoint_class</span><span class="p">(</span><span class="n">address</span><span class="p">,</span> <span class="n">port</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">opt</span> <span class="ow">in</span> <span class="p">(</span><span class="s">&#39;-h&#39;</span><span class="p">,</span> <span class="s">&#39;--help&#39;</span><span class="p">):</span>
<span class="n">args</span><span class="p">[</span><span class="s">&#39;print_help&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">True</span>
<span class="c"># translates our args dict into a named tuple</span>
<span class="n">Args</span> <span class="o">=</span> <span class="n">collections</span><span class="o">.</span><span class="n">namedtuple</span><span class="p">(</span><span class="s">&#39;Args&#39;</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
<span class="k">return</span> <span class="n">Args</span><span class="p">(</span><span class="o">**</span><span class="n">args</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">args</span> <span class="o">=</span> <span class="n">parse</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">:])</span>
<span class="k">except</span> <span class="ne">ValueError</span> <span class="k">as</span> <span class="n">exc</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="n">exc</span><span class="p">)</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">if</span> <span class="n">args</span><span class="o">.</span><span class="n">print_help</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="n">HELP_TEXT</span><span class="p">)</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">()</span>
<span class="k">print</span><span class="p">(</span><span class="s">&#39;Downloading </span><span class="si">%s</span><span class="s"> descriptor from </span><span class="si">%s</span><span class="s">:</span><span class="si">%s</span><span class="s">...</span><span class="se">\n</span><span class="s">&#39;</span> <span class="o">%</span> <span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">descriptor_type</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">download_from</span><span class="o">.</span><span class="n">address</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">download_from</span><span class="o">.</span><span class="n">port</span><span class="p">))</span>
<span class="n">desc</span> <span class="o">=</span> <span class="bp">None</span>
<span class="k">if</span> <span class="n">args</span><span class="o">.</span><span class="n">descriptor_type</span> <span class="ow">in</span> <span class="p">(</span><span class="s">&#39;server&#39;</span><span class="p">,</span> <span class="s">&#39;extrainfo&#39;</span><span class="p">):</span>
<span class="k">if</span> <span class="n">args</span><span class="o">.</span><span class="n">descriptor_type</span> <span class="o">==</span> <span class="s">&#39;server&#39;</span><span class="p">:</span>
<span class="n">download_func</span> <span class="o">=</span> <span class="n">stem</span><span class="o">.</span><span class="n">descriptor</span><span class="o">.</span><span class="n">remote</span><span class="o">.</span><span class="n">get_server_descriptors</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">download_func</span> <span class="o">=</span> <span class="n">stem</span><span class="o">.</span><span class="n">descriptor</span><span class="o">.</span><span class="n">remote</span><span class="o">.</span><span class="n">get_extrainfo_descriptors</span>
<span class="n">desc</span> <span class="o">=</span> <span class="n">download_func</span><span class="p">(</span>
<span class="n">fingerprints</span> <span class="o">=</span> <span class="p">[</span><span class="n">args</span><span class="o">.</span><span class="n">fingerprint</span><span class="p">],</span>
<span class="n">endpoints</span> <span class="o">=</span> <span class="p">[</span><span class="n">args</span><span class="o">.</span><span class="n">download_from</span><span class="p">],</span>
<span class="p">)</span><span class="o">.</span><span class="n">run</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">elif</span> <span class="n">args</span><span class="o">.</span><span class="n">descriptor_type</span> <span class="o">==</span> <span class="s">&#39;consensus&#39;</span><span class="p">:</span>
<span class="k">for</span> <span class="n">consensus_desc</span> <span class="ow">in</span> <span class="n">stem</span><span class="o">.</span><span class="n">descriptor</span><span class="o">.</span><span class="n">remote</span><span class="o">.</span><span class="n">get_consensus</span><span class="p">(</span><span class="n">endpoints</span> <span class="o">=</span> <span class="p">[</span><span class="n">args</span><span class="o">.</span><span class="n">download_from</span><span class="p">]):</span>
<span class="k">if</span> <span class="n">consensus_desc</span><span class="o">.</span><span class="n">fingerprint</span> <span class="o">==</span> <span class="n">args</span><span class="o">.</span><span class="n">fingerprint</span><span class="p">:</span>
<span class="n">desc</span> <span class="o">=</span> <span class="n">consensus_desc</span>
<span class="k">break</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">desc</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="s">&#39;Unable to find a descriptor for </span><span class="si">%s</span><span class="s"> in the consensus&#39;</span> <span class="o">%</span> <span class="n">args</span><span class="o">.</span><span class="n">fingerprint</span><span class="p">)</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="s">&quot;&#39;</span><span class="si">%s</span><span class="s">&#39; is not a recognized descriptor type, options are: </span><span class="si">%s</span><span class="s">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">descriptor_type</span><span class="p">,</span> <span class="s">&#39;, &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">VALID_TYPES</span><span class="p">)))</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">desc</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">&#39;__main__&#39;</span><span class="p">:</span>
<span class="n">main</span><span class="p">()</span>
</pre></div>
</div>
<div class="highlight-python"><pre>% python download_descriptor.py --type consensus --dirport 128.31.0.34:9131
Downloading consensus descriptor from 128.31.0.34:9131...
r moria1 lpXfw1/+uGEym58asExGOXAgzjE IpcU7dolas8+Q+oAzwgvZIWx7PA 2018-05-23 02:41:25 128.31.0.34 9101 9131
s Authority Fast Running Stable V2Dir Valid
v Tor 0.3.3.5-rc-dev
pr Cons=1-2 Desc=1-2 DirCache=1-2 HSDir=1-2 HSIntro=3-4 HSRend=1-2 Link=1-5 LinkAuth=1,3 Microdesc=1-2 Relay=1-2
w Bandwidth=20 Unmeasured=1
p reject 1-65535</pre>
</div>
</div>
</div>
<div class="bottomnav">
</div>
<div class="footer">
</div>
</body>
</html>