emdee/phantompy
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: emdee:48a4b37b76c4402b503bcfc539b49ae5df59b9be
Branches Tags
emdee:master
..
compare: emdee:b0a18addb3cfa8b0934f8584578aebf859f77acc
Branches Tags
emdee:master

No commits in common. "48a4b37b76c4402b503bcfc539b49ae5df59b9be" and "b0a18addb3cfa8b0934f8584578aebf859f77acc" have entirely different histories.
48a4b37b76 ... b0a18addb3

14 changed files with 149 additions and 487 deletions
Show Stats Expand all files Collapse all files

47
.github/workflows/test.yml vendored
View File

@ -1,47 +0,0 @@
name: test
on: [push]
jobs:
ci:
name: Python-${{ matrix.python }} ${{ matrix.qt.qt_api }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
qt:
- package: PyQt5
qt_api: "pyqt5"
- package: PyQt6
qt_api: "pyqt6"
- package: PySide2
qt_api: "pyside2"
- package: PySide6
qt_api: "pyside6"
python: [3.6, 3.7, 3.8, 3.9]
steps:
- name: Checkout
uses: actions/checkout@v1
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python }}
architecture: x64
- name: Install pipenv
run: |
python -m pip install --upgrade pipenv wheel
- name: Install dependencies
run: |
pipenv install --dev
pipenv run pip install ${{ matrix.qt.package }} pytest
- name: Install Libxcb dependencies
run: |
sudo apt-get update
sudo apt-get install '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev
- name: Run headless test
uses: GabrielBB/xvfb-action@v1
env:
QT_API: ${{ matrix.qt.qt_api }}
with:
run: pipenv run py.test --forked -v

16
Pipfile
View File

@ -1,16 +0,0 @@
[[source]]
url = "https://pypi.org/simple"
name = "pypi"
verify_ssl = true
[dev-packages]
atomicwrites = "*"
pytest = "*"
pytest-forked = "*"
pytest-raises = "*"
[packages]
[dev-packages.phantomjs]
editable = true
path = "."

24
README.md
View File

@ -4,9 +4,8 @@ A simple replacement for phantomjs using PyQt.
This code is based on a brilliant idea of
[Michael Franzl](https://gist.github.com/michaelfranzl/91f0cc13c56120391b949f885643e974/raw/a0601515e7a575bc4c7d4d2a20973b29b6c6f2df/phantom.py)
that he wrote up in his blog:
* https://blog.michael.franzl.name/2017/10/16/phantomjs-alternative-write-short-pyqt-scripts-instead-phantom-py/
* https://blog.michael.franzl.name/2017/10/16/phantom-py/
that he wrote up in his
[blog](https://blog.michael.franzl.name/2017/10/16/phantom-py/index.html)
## Features
@ -27,22 +26,17 @@ way of knowing when that script has finished doing its work. For this
reason, the external script should execute at the end
```console.log("__PHANTOM_PY_DONE__");``` when done. This will trigger
the PDF generation or the file saving, after which phantompy will exit.
If you do not want to run any javascipt file, this trigger is provided
in the code by default.
It is important to remember that since you're just running WebKit, you can
use everything that WebKit supports, including the usual JS client
libraries, CSS, CSS @media types, etc.
Qt picks up proxies from the environment, so this will respect
```https_proxy``` or ```http_proxy``` if set.
## Dependencies
* Python3
* PyQt5 (this should work with PySide2 and PyQt6 - let us know.)
* [qasnyc](https://github.com/CabbageDevelopment/qasync) for the
standalone program ```qasync_phantompy.py```
standalone program ```qasync_lookup.py```
## Standalone
@ -63,10 +57,10 @@ for the PyQt ```app.exec``` and the exiting of the program.
We've decided to use the best of the shims that merge the Python
```asyncio``` and Qt event loops:
[qasyc](https://github.com/CabbageDevelopment/qasync). This is seen as
the successor to the sorta abandoned [quamash](https://github.com/harvimt/quamash).
the successor to the sorta abandoned[quamash](https://github.com/harvimt/quamash).
The code is based on a
[comment](https://github.com/CabbageDevelopment/qasync/issues/35#issuecomment-1315060043)
by [Alex March](https://github.com/hosaka) who's excellent code helped me.
by [Alex Marcha](https://github.com/hosaka) who's excellent code helped me.
As this is my first use of ```asyncio``` and ```qasync``` I may have
introduced some errors and it may be improved on, but it works, and
it not a monolithic Qt program, so it can be used as a library.
@ -79,11 +73,9 @@ The standalone program is ```quash_phantompy.py```
### Arguments
```
--js_input (optional) Path and name of a JavaScript file to execute on the HTML
--html_output <html-file> (optional) Path a HTML output file to generate after JS is applied
--pdf_output <pdf-file> (optional) Path and name of PDF file to generate after JS is applied
--log_level 10=debug 20=info 30=warn 40=error
html_or_url - required argument, a http(s) URL or a path to a local file.
<url> Can be a http(s) URL or a path to a local file
<pdf-file> Path and name of PDF file to generate
[<javascript-file>] (optional) Path and name of a JavaScript file to execute
```
Setting ```DEBUG=1``` in the environment will give debugging messages
on ```stderr```.

0
__init__.py Normal file
View File

10
phantompy/__main__.py → __main__.py
View File

@ -1,13 +1,10 @@
#!/usr/local/bin/python3.sh
# -*-mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*
from __future__ import absolute_import
import sys
from .qasync_phantompy import iMain
from qasync_phantompy import iMain
try:
from .support_phantompy import vsetup_logging
from support_phantompy import vsetup_logging
d = int(os.environ.get('DEBUG', 0))
if d > 0:
vsetup_logging(10, stream=sys.stderr)
@ -16,5 +13,4 @@ try:
vsetup_logging(log_level, logfile='', stream=sys.stderr)
except: pass
if __name__ == '__main__':
iMain(sys.argv[1:])
iMain(sys.argv[1:], bgui=False)

22
appveyor.yml
View File

@ -1,22 +0,0 @@
environment:
matrix:
- PYTHON: "C:\\Python36"
- PYTHON: "C:\\Python37"
- PYTHON: "C:\\Python38"
- PYTHON: "C:\\Python39"
init:
- set PATH=%PYTHON%;%PYTHON%\Scripts;%PATH%
install:
- pip install pipenv
- pipenv install --dev
- pipenv run pip install PyQt5 PySide2
# FIX: colorama not installed by pipenv
- pipenv run pip install colorama
build: off
test_script:
- set QT_API=PyQt5&& pipenv run py.test -v
- set QT_API=PySide2&& pipenv run py.test -v

155
phantompy/phantompy.py → phantompy.py
View File

@ -13,18 +13,17 @@ replacement for other bulky headless browser frameworks.
If you have a display attached:
./phantom.py [--pdf_output <pdf-file>] [--js_input <javascript-file>] <url-or-html-file>
./phantom.py <url> <pdf-file> [<javascript-file>]
If you don't have a display attached (i.e. on a remote server), you can use
xvfb-run, or don't add --show_gui - it should work without a display.
If you don't have a display attached (i.e. on a remote server):
xvfb-run ./phantom.py <url> <pdf-file> [<javascript-file>]
Arguments:
[--pdf_output <pdf-file>] (optional) Path and name of PDF file to generate
[--html_output <html-file>] (optional) Path and name of HTML file to generate
[--js_input <javascript-file>] (optional) Path and name of a JavaScript file to execute
--log_level 10=debug 20=info 30=warn 40=error
<url> Can be a http(s) URL or a path to a local file
<pdf-file> Path and name of PDF file to generate
[<javascript-file>] (optional) Path and name of a JavaScript file to execute
## Features
@ -56,9 +55,6 @@ CSS @media types, etc.
* Python3
* PyQt5
* [qasnyc](https://github.com/CabbageDevelopment/qasync) for the
standalone program ```qasnyc_phantompy.py```
* xvfb (optional for display-less machines)
Installation of dependencies in Debian Stretch is easy:
@ -118,20 +114,23 @@ IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"""
import importlib
import sys
import os
import sys # noqa
import traceback
import atexit
import time
from qasync import QtModuleName
from qasync.QtCore import QUrl
from PyQt5.QtCore import QUrl
from PyQt5.QtCore import QTimer
from PyQt5.QtWidgets import QApplication
from PyQt5.QtPrintSupport import QPrinter
from PyQt5.QtWebEngineWidgets import QWebEnginePage
QPrinter = importlib.import_module(QtModuleName + ".QtPrintSupport.QPrinter", package=QtModuleName)
QWebEnginePage = importlib.import_module(QtModuleName + ".QtWebEngineWidgets.QWebEnginePage", package=QtModuleName)
from support_phantompy import vsetup_logging
global LOG
import logging
import warnings
warnings.filterwarnings('ignore')
LOG = logging.getLogger()
@ -161,38 +160,35 @@ def prepare(sdir='/tmp'):
class Render(QWebEnginePage):
def __init__(self, app, do_print=False, do_save=True):
app.ldone = []
self._app = app
self.do_print = do_print
self.do_save = do_save
self.percent = 0
self.uri = None
self.jsfile = None
self.htmlfile = None
self.pdffile = None
QWebEnginePage.__init__(self)
app.ldone = []
self._app = app
self.do_print = do_print
self.do_save = do_save
self.percent = 0
self.uri = None
self.jsfile = None
self.outfile = None
QWebEnginePage.__init__(self)
def run(self, url, pdffile, htmlfile, jsfile):
def run(self, url, outfile, jsfile):
self._app.lstart.append(id(self))
self.percent = 10
self.uri = url
self.jsfile = jsfile
self.htmlfile = htmlfile
self.pdffile = pdffile
self.outfile = pdffile or htmlfile
LOG.debug(f"phantom.py: URL={url} htmlfile={htmlfile} pdffile={pdffile} JSFILE={jsfile}")
self.outfile = outfile
LOG.debug(f"phantom.py: URL={url} OUTFILE={outfile} JSFILE={jsfile}")
qurl = QUrl.fromUserInput(url)
# The PDF generation only happens when the special string __PHANTOM_PY_DONE__
# is sent to console.log(). The following JS string will be executed by
# default, when no external JavaScript file is specified.
self.js_contents = "setTimeout(function() { console.log('__PHANTOM_PY_DONE__') }, 5000);"
self.js_contents = "setTimeout(function() { console.log('__PHANTOM_PY_DONE__') }, 5000);";
if jsfile:
try:
with open(self.jsfile, 'rt') as f:
self.js_contents = f.read()
except Exception as e: # noqa
except Exception as e:
LOG.exception(f"error reading jsfile {self.jsfile}")
self.loadFinished.connect(self._loadFinished)
@ -202,46 +198,45 @@ class Render(QWebEnginePage):
LOG.debug(f"phantom.py: loading 10")
def _onConsoleMessage(self, *args):
if len(args) > 3:
level, txt, lineno, filename = args
else:
level = 1
txt, lineno, filename = args
LOG.debug(f"CONSOLE {lineno} {txt} {filename}")
if "__PHANTOM_PY_DONE__" in txt:
self.percent = 40
# If we get this magic string, it means that the external JS is done
if self.do_save:
self.toHtml(self._html_callback)
return
# drop through
txt = "__PHANTOM_PY_SAVED__"
if "__PHANTOM_PY_SAVED__" in txt:
self.percent = 50
if self.do_print:
self._print()
return
txt = "__PHANTOM_PY_PRINTED__"
if "__PHANTOM_PY_PRINTED__" in txt:
self.percent = 60
self._exit(level)
if len(args) > 3:
level, txt, lineno, filename = args
else:
level = 1
txt, lineno, filename = args
LOG.debug(f"CONSOLE {lineno} {txt} {filename}")
if "__PHANTOM_PY_DONE__" in txt:
self.percent = 40
# If we get this magic string, it means that the external JS is done
if self.do_save:
self.toHtml(self._html_callback)
return
# drop through
txt = "__PHANTOM_PY_SAVED__"
if "__PHANTOM_PY_SAVED__" in txt:
self.percent = 50
if self.do_print:
self._print()
return
txt = "__PHANTOM_PY_PRINTED__"
if "__PHANTOM_PY_PRINTED__" in txt:
self.percent = 60
self._exit(level)
def _loadFinished(self, result):
# RenderProcessTerminationStatus ?
self.percent = 30
LOG.info(f"phantom.py: _loadFinished {result} {self.percent}")
LOG.debug(f"phantom.py: Evaluating JS from {self.jsfile}")
self.runJavaScript("document.documentElement.contentEditable=true")
self.runJavaScript(self.js_contents)
self.percent = 30
LOG.info(f"phantom.py: _loadFinished {result} {self.percent}")
LOG.debug(f"phantom.py: Evaluating JS from {self.jsfile}")
self.runJavaScript("document.documentElement.contentEditable=true")
self.runJavaScript(self.js_contents)
def _html_callback(self, *args):
"""print(self, QPrinter, Callable[[bool], None])"""
if type(args[0]) is str:
self._save(args[0])
self._onConsoleMessage(0, "__PHANTOM_PY_SAVED__", 0, '')
self._onConsoleMessage(0, "__PHANTOM_PY_SAVED__", 0 , '')
def _save(self, html):
sfile = self.htmlfile
sfile = self.outfile.replace('.pdf','.html')
# CompleteHtmlSaveFormat SingleHtmlSaveFormat MimeHtmlSaveFormat
with open(sfile, 'wt') as ofd:
ofd.write(html)
@ -249,19 +244,20 @@ class Render(QWebEnginePage):
def _printer_callback(self, *args):
"""print(self, QPrinter, Callable[[bool], None])"""
# print(f"_printer_callback {self.outfile} {args}")
if args[0] is False:
i = 1
else:
i = 0
self._onConsoleMessage(i, "__PHANTOM_PY_PRINTED__", 0, '')
self._onConsoleMessage(i, "__PHANTOM_PY_PRINTED__", 0 , '')
def _print(self):
sfile = self.pdffile
sfile = self.outfile.replace('.html', '.pdf')
printer = QPrinter()
printer.setPageMargins(10, 10, 10, 10, QPrinter.Millimeter)
printer.setPaperSize(QPrinter.A4)
printer.setCreator("phantom.py by Michael Karl Franzl")
printer.setOutputFormat(QPrinter.PdfFormat)
printer.setOutputFormat(QPrinter.PdfFormat);
printer.setOutputFileName(sfile)
self.print(printer, self._printer_callback)
LOG.debug("phantom.py: Printed")
@ -271,3 +267,26 @@ class Render(QWebEnginePage):
LOG.debug(f"phantom.py: Exiting with val {val}")
# threadsafe?
self._app.ldone.append(self.uri)
def omain(app, largs):
if (len(largs) < 2):
LOG.info("USAGE: ./phantom.py <url> <pdf-file> [<javascript-file>]")
return -1
url = largs[0]
outfile = largs[1]
jsfile = largs[2] if len(largs) > 2 else None
ilen = 1
r = Render(app, do_print=False, do_save=True)
r.run(url, outfile, jsfile)
for i in range(1, 120):
app.processEvents()
print(f"{app.ldone} {i}")
if len(app.ldone) == ilen:
print(f"{app.ldone} found {ilen}")
app.exit()
return r
time.sleep(1)
return r

3
phantompy/__init__.py
View File

@ -1,3 +0,0 @@
# -*-mode: python; indent-tabs-mode: nil; py-indent-offset: 2; coding: utf-8 -*-
__version__ = "0.1.0"

84
phantompy/lookupdns.py
View File

@ -1,84 +0,0 @@
#!/usr/local/bin/python3.sh
# -*-mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*
"""
Looks for urls https://dns.google/resolve?
https://dns.google/resolve?name=domain.name&type=TXT&cd=true&do=true
and parses them to extract a magic field.
A good example of how you can parse json embedded in HTML with phantomjs.
"""
import sys
import os
from phantompy import Render
global LOG
import logging
import warnings
warnings.filterwarnings('ignore')
LOG = logging.getLogger()
class LookFor(Render):
def __init__(self, app, do_print=True, do_save=False):
app.lfps = []
self._app = app
self.do_print = do_print
self.do_save = do_save
self.progress = 0
self.we_run_this_tor_relay = None
Render.__init__(self, app, do_print, do_save)
def _exit(self, val):
Render._exit(self, val)
self.percent = 100
LOG.debug(f"phantom.py: Exiting with val {val}")
i = self.uri.find('name=')
fp = self.uri[i+5:]
i = fp.find('.')
fp = fp[:i]
# threadsafe?
self._app.lfps.append(fp)
def _html_callback(self, *args):
"""print(self, QPrinter, Callable[[bool], None])"""
if type(args[0]) is str:
self._save(args[0])
i = self.ilookfor(args[0])
self._onConsoleMessage(i, "__PHANTOM_PY_SAVED__", 0 , '')
def ilookfor(self, html):
import json
marker = '<pre style="word-wrap: break-word; white-space: pre-wrap;">'
if marker not in html: return -1
i = html.find(marker) + len(marker)
html = html[i:]
assert html[0] == '{', html
i = html.find('</pre')
html = html[:i]
assert html[-1] == '}', html
LOG.debug(f"Found {len(html)} json")
o = json.loads(html)
if "Answer" not in o.keys() or type(o["Answer"]) != list:
LOG.warn(f"FAIL {self.uri}")
return 1
for elt in o["Answer"]:
assert type(elt) == dict, elt
assert 'type' in elt, elt
if elt['type'] != 16: continue
assert 'data' in elt, elt
if elt['data'] == 'we-run-this-tor-relay':
LOG.info(f"OK {self.uri}")
self.we_run_this_tor_relay = True
return 0
self.we_run_this_tor_relay = False
LOG.warn(f"BAD {self.uri}")
return 2
def _loadFinished(self, result):
LOG.debug(f"phantom.py: Loading finished {self.uri}")
self.toHtml(self._html_callback)

75
phantompy/qasync_phantompy.py → qasync_phantompy.py
View File

@ -1,30 +1,25 @@
#!/usr/local/bin/python3.sh
# -*-mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*
import asyncio
import os
import sys
import os
import qasync
import asyncio
import time
import random
# let qasync figure out what Qt we are using - we dont care
from qasync import QApplication, QEventLoop, QtWidgets
from PyQt5 import QtWidgets
from PyQt5.QtWidgets import (QProgressBar, QWidget, QVBoxLayout)
from phantompy import Render
# if you want an example of looking for things in downloaded HTML:
# from lookupdns import LookFor as Render
from support_phantompy import omain_argparser, vsetup_logging
global LOG
import logging
import warnings
warnings.filterwarnings('ignore')
LOG = logging.getLogger()
try:
import shtab
except:
shtab = None
class Widget(QtWidgets.QWidget):
def __init__(self):
QtWidgets.QWidget.__init__(self)
@ -32,7 +27,7 @@ class Widget(QtWidgets.QWidget):
box = QtWidgets.QHBoxLayout()
self.setLayout(box)
box.addWidget(self._label)
self.progress = QtWidgets.QProgressBar()
self.progress = QProgressBar()
self.progress.setRange(0, 99)
box.addWidget(self.progress)
@ -42,17 +37,13 @@ class Widget(QtWidgets.QWidget):
self.progress.setValue(int(text))
class ContextManager:
def __init__(self) -> None:
self._seconds = 0
async def __aenter__(self):
LOG.debug("ContextManager enter")
return self
async def __aexit__(self, *args):
LOG.debug("ContextManager exit")
async def tick(self):
await asyncio.sleep(1)
self._seconds += 1
@ -72,25 +63,12 @@ async def main(widget, app, ilen):
app.exit()
# raise asyncio.CancelledError
return
LOG.debug(f"{app.ldone} {seconds}")
except asyncio.CancelledError as ex: # noqa
LOG.debug(f"{app.ldone} {perc} {seconds}")
except asyncio.CancelledError as ex:
LOG.debug("Task cancelled")
def iMain(largs):
parser = omain_argparser()
if shtab:
shtab.add_argument_to(parser, ["-s", "--print-completion"]) # magic!
oargs = parser.parse_args(largs)
bgui = oargs.show_gui
try:
d = int(os.environ.get('DEBUG', 0))
if d > 0:
oargs.log_level = 10
vsetup_logging(oargs.log_level, logfile='', stream=sys.stderr)
except: pass
app = QApplication([])
def iMain(largs, bgui=True):
app = QtWidgets.QApplication([])
app.lstart = []
if bgui:
widget = Widget()
@ -99,19 +77,17 @@ def iMain(largs):
else:
widget = None
loop = QEventLoop(app)
loop = qasync.QEventLoop(app)
asyncio.set_event_loop(loop)
url = oargs.html_url
htmlfile = oargs.html_output
pdffile = oargs.html_output
jsfile = oargs.js_input
largs = sys.argv[1:]
url = largs[0]
outfile = largs[1]
jsfile = largs[2] if len(largs) > 2 else None
# run only starts the url loading
r = Render(app,
do_print=True if pdffile else False,
do_save=True if htmlfile else False)
r = Render(app, do_print=False, do_save=True)
uri = url.strip()
r.run(uri, pdffile, htmlfile, jsfile)
r.run(uri, outfile, jsfile)
LOG.debug(f"{r.percent} {app.lstart}")
LOG.info(f"queued {len(app.lstart)} urls")
@ -125,4 +101,15 @@ def iMain(largs):
loop.run_until_complete(asyncio.gather(*tasks))
if __name__ == '__main__':
iMain(sys.argv[1:])
try:
from exclude_badExits import vsetup_logging
d = int(os.environ.get('DEBUG', 0))
if d > 0:
vsetup_logging(10, stream=sys.stderr)
else:
vsetup_logging(20, stream=sys.stderr)
vsetup_logging(log_level, logfile='', stream=sys.stderr)
except: pass
iMain(sys.argv[1:], bgui=False)

58
setup.cfg
View File

@ -1,58 +0,0 @@
[metadata]
classifiers =
License :: OSI Approved
License :: OSI Approved :: BSD 1-clause
Intended Audience :: Web Developers
Operating System :: Microsoft :: Windows
Operating System :: POSIX :: BSD :: FreeBSD
Operating System :: POSIX :: Linux
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: Implementation :: CPython
Framework :: AsyncIO
[options]
zip_safe = false
python_requires = ~=3.6
packages = find:
include_package_data = false
install_requires =
qasync
cryptography
rsa
stem
[options.entry_points]
console_scripts =
phantompy = phantompy.__main__:iMain
[easy_install]
zip_ok = false
[flake8]
jobs = 1
max-line-length = 88
ignore =
E111
E114
E128
E225
E225
E261
E302
E305
E402
E501
E502
E541
E701
E704
E722
E741
F508
F541
W503

44
setup.py
View File

@ -1,44 +0,0 @@
# -*-mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*
import re
from setuptools import setup
with open("qasync/__init__.py") as f:
version = re.search(r'__version__\s+=\s+"(.*)"', f.read()).group(1)
long_description = "\n\n".join([
open("README.md").read(),
])
if __name__ == '__main__':
setup(
name="phantompy",
version=__version__,
description="""A simple replacement for phantomjs using PyQt""",
long_description=long_description,
author="Michael Franzl (originally)",
author_email='',
license="1clause BSD",
packages=['phantompy'],
# url="",
# download_url="https://",
keywords=['JavaScript', 'phantomjs', 'asyncio'],
# maybe less - nothing fancy
python_requires="~=3.6",
# probably works on PyQt6 and PySide2 but untested
# https://github.com/CabbageDevelopment/qasync/
install_requires=['qasync',
'PyQt5'],
entry_points={
'console_scripts': ['phantompy = phantompy.__main__:iMain', ]},
classifiers=[
'Development Status :: 4 - Beta',
'Environment :: Console',
'Intended Audience :: Developers',
'Intended Audience :: Web Developers',
'Natural Language :: English',
'Operating System :: OS Independent',
'Programming Language :: Python :: 3',
'Topic :: Software Development :: Documentation',
],
)

42
phantompy/support_phantompy.py → support_phantompy.py
View File

@ -1,22 +1,20 @@
#!/usr/local/bin/python3.sh
# -*-mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*
import argparse
import os
import sys
import os
try:
if 'COLOREDLOGS_LEVEL_STYLES' not in os.environ:
os.environ['COLOREDLOGS_LEVEL_STYLES'] = 'spam=22;debug=28;verbose=34;notice=220;warning=202;success=118,bold;error=124;critical=background=red'
# https://pypi.org/project/coloredlogs/
import coloredlogs
except ImportError:
except ImportError as e:
coloredlogs = False
global LOG
import logging
import warnings
warnings.filterwarnings('ignore')
LOG = logging.getLogger()
@ -25,7 +23,7 @@ def vsetup_logging(log_level, logfile='', stream=sys.stdout):
add = True
# stem fucks up logging
# from stem.util import log
from stem.util import log
logging.getLogger('stem').setLevel(30)
logging._defaultFormatter = logging.Formatter(datefmt='%m-%d %H:%M:%S')
@ -81,37 +79,3 @@ def vsetup_logging(log_level, logfile='', stream=sys.stdout):
'NOTSET': logging.NOTSET,
}
def omain_argparser(_=None):
try:
from OpenSSL import SSL
lCAfs = SSL._CERTIFICATE_FILE_LOCATIONS
except:
lCAfs = []
CAfs = []
for elt in lCAfs:
if os.path.exists(elt):
CAfs.append(elt)
if not CAfs:
CAfs = ['']
parser = argparse.ArgumentParser(add_help=True,
epilog=__doc__)
parser.add_argument('--https_cafile', type=str,
help="Certificate Authority file (in PEM) (unused)",
default=CAfs[0])
parser.add_argument('--log_level', type=int, default=20,
help="10=debug 20=info 30=warn 40=error")
parser.add_argument('--js_input', type=str, default='',
help="Operate on the HTML file with javascript")
parser.add_argument('--html_output', type=str, default='',
help="Write loaded and javascripted result to a HTML file")
parser.add_argument('--pdf_output', type=str, default='',
help="Write loaded and javascripted result to a PDF file")
parser.add_argument('--show_gui', type=bool, default=False, store_action=True,
help="show a progress meter that doesn't work")
parser.add_argument('html_url', type=str, nargs='?',
required=True,
help='html file or url')
return parser

22
tests/conftest.py
View File

@ -1,22 +0,0 @@
# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
# (c) 2018 Gerard Marull-Paretas <gerard@teslabs.com>
# (c) 2014 Mark Harviston <mark.harviston@gmail.com>
# (c) 2014 Arve Knudsen <arve.knudsen@gmail.com>
# BSD License
# phantompy test - just test qasync for now
import os
import logging
from pytest import fixture
logging.basicConfig(
level=logging.DEBUG, format="%(asctime)s - %(levelname)s - %(name)s - %(message)s"
)
@fixture(scope="session")
def application():
from phantompy.qasync_phantompy import QApplication
return QApplication([])