Oops

README.md

@@ -2,13 +2,7 @@
 
 Toxygen is powerful cross-platform [Tox](https://tox.chat/) client written in pure Python3.
 
-[![Release](https://img.shields.io/github/release/toxygen-project/toxygen.svg?style=flat)](https://github.com/toxygen-project/toxygen/releases/latest)
+### [Install](/docs/install.md) - [Contribute](/docs/contributing.md) - [Plugins](/docs/plugins.md) - [Compile](/docs/compile.md) - [Contact](/docs/contact.md)
-[![Stars](https://img.shields.io/github/stars/toxygen-project/toxygen.svg?style=flat)](https://github.com/toxygen-project/toxygen/stargazers)
-[![Open issues](https://img.shields.io/github/issues/toxygen-project/toxygen.svg?style=flat)](https://github.com/toxygen-project/toxygen/issues)
-[![License](https://img.shields.io/badge/license-GPLv3-blue.svg?style=flat)](https://raw.githubusercontent.com/toxygen-project/toxygen/master/LICENSE.md)
-[![Build Status](https://travis-ci.org/toxygen-project/toxygen.svg?branch=master)](https://travis-ci.org/toxygen-project/toxygen)
-
-### [Install](/docs/install.md) - [Contribute](/docs/contributing.md) - [Plugins](/docs/plugins.md) - [Compile](/docs/compile.md) - [Contact](/docs/contact.md) - [Updater](https://github.com/toxygen-project/toxygen_updater)
 
 ### Supported OS: Linux and Windows
 
@@ -43,22 +37,23 @@ Toxygen is powerful cross-platform [Tox](https://tox.chat/) client written in pu
 - Changing nospam
 - File resuming
 - Read receipts
-
+- NGC groups
-### Downloads
-[Releases](https://github.com/toxygen-project/toxygen/releases)
-
-[Download last stable version](https://github.com/toxygen-project/toxygen/archive/master.zip)
-
-[Download develop version](https://github.com/toxygen-project/toxygen/archive/develop.zip)
 
 ### Screenshots
 *Toxygen on Ubuntu and Windows*
 ![Ubuntu](/docs/ubuntu.png)
 ![Windows](/docs/windows.png)
 
-### Docs
+## Forked
-[Check /docs/ for more info](/docs/)
 
-Also visit [pythonhosted.org/Toxygen/](http://pythonhosted.org/Toxygen/)
+This hard-forked from https://github.com/toxygen-project/toxygen
+```next_gen``` branch.
 
-[Wiki](https://wiki.tox.chat/clients/toxygen)
+https://git.plastiras.org/emdee/toxygen_wrapper needs packaging
+is making a dependency. Just download it and copy the two directories
+```wrapper``` and ```wrapper_tests``` into ```toxygen/toxygen```.
+
+See ToDo.md to the current ToDo list.
+
+Work on this project is suspended until the
+[MultiDevice](https://git.plastiras.org/emdee/tox_profile/wiki/MultiDevice-Announcements-POC) problem is solved. Fork me!

ToDo.md

@@ -0,0 +1,52 @@
+# Toxygen ToDo List
+
+## Bugs
+
+1. There is an agravating bug  where new messages are not put in the
+   current window, and a messages waiting indicator appears. You have
+   to focus out of the window and then back in the window.
+
+
+
+## Fix history
+
+The code is in there but it's not working.
+
+## Fix Audio
+
+The code is in there but it's not working. It looks like audio input
+is working but not output. The code is all in there; I may have broken
+it trying to wire up the ability to set the audio device from the
+command line.
+
+## Fix Video
+
+The code is in there but it's not working.  I may have broken it
+trying to wire up the ability to set the video device from the command
+line.
+
+## Groups
+
+1. peer_id There has been a change of API on a field named
+   ```group.peer_id``` The code is broken in places because I have not
+   seen the path to change from the old API ro the new one.
+
+
+## Plugin system
+
+1. Needs better documentation and checking.
+
+2. There's something broken in the way some of them plug into Qt menus.
+
+3. Should the plugins be in toxygen or a separate repo?
+
+4. There needs to be a uniform way for plugins to wire into callbacks.
+
+## check toxygen_wrapper
+
+1. I've broken out toxygen_wrapper to be standalone,
+   https://git.plastiras.org/emdee/toxygen_wrapper but the tox.py
+   needs each call double checking.
+
+2. https://git.plastiras.org/emdee/toxygen_wrapper needs packaging
+   and making a dependency.

setup.py

@@ -12,17 +12,14 @@ version = main.__version__ + '.0'
 
 
 if system() == 'Windows':
-    MODULES = ['PyQt5', 'PyAudio', 'numpy', 'opencv-python', 'pydenticon']
+    MODULES = ['PyQt5', 'PyAudio', 'numpy', 'opencv-python', 'pydenticon', 'cv2']
 else:
-    MODULES = []
+    MODULES = ['pydenticon']
+    MODULES.append('PyQt5')
     try:
         import pyaudio
     except ImportError:
         MODULES.append('PyAudio')
-    try:
-        import PyQt5
-    except ImportError:
-        MODULES.append('PyQt5')
     try:
         import numpy
     except ImportError:
@@ -32,9 +29,13 @@ else:
     except ImportError:
         MODULES.append('opencv-python')
     try:
-        import pydenticon
+        import coloredlogs
     except ImportError:
-        MODULES.append('pydenticon')
+        MODULES.append('coloredlogs')
+    try:
+        import pyqtconsole
+    except ImportError:
+        MODULES.append('pyqtconsole')
 
 
 def get_packages():
@@ -42,10 +43,8 @@ def get_packages():
     for root, dirs, files in os.walk(directory):
         packages = map(lambda d: 'toxygen.' + d, dirs)
         packages = ['toxygen'] + list(packages)
-
         return packages
 
-
 class InstallScript(install):
     """This class configures Toxygen after installation"""
 
@@ -72,22 +71,23 @@ setup(name='Toxygen',
       version=version,
       description='Toxygen - Tox client',
       long_description='Toxygen is powerful Tox client written in Python3',
-      url='https://github.com/toxygen-project/toxygen/',
+      url='https://git.plastiras.org/emdee/toxygen/',
-      keywords='toxygen tox messenger',
+      keywords='toxygen Tox messenger',
       author='Ingvar',
-      maintainer='Ingvar',
+      maintainer='',
       license='GPL3',
       packages=get_packages(),
       install_requires=MODULES,
       include_package_data=True,
       classifiers=[
           'Programming Language :: Python :: 3 :: Only',
-          'Programming Language :: Python :: 3.5',
+          'Programming Language :: Python :: 3.9',
-          'Programming Language :: Python :: 3.6',
       ],
       entry_points={
           'console_scripts': ['toxygen=toxygen.main:main']
       },
       cmdclass={
           'install': InstallScript
-      })
+      },
+      zip_safe=False
+      )

toxygen/av/calls.py

@@ -1,21 +1,54 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 import pyaudio
 import time
 import threading
-from wrapper.toxav_enums import *
-import cv2
 import itertools
-import numpy as np
+
+from wrapper.toxav_enums import *
 from av import screen_sharing
 from av.call import Call
 import common.tox_save
 
+from utils import ui as util_ui
+import wrapper_tests.support_testing as ts
+from middleware.threads import invoke_in_main_thread
+from main import sleep
+from middleware.threads import BaseThread
+
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
+# callbacks can be called in any thread so were being careful
+def LOG_ERROR(l): print('EROR< '+l)
+def LOG_WARN(l):  print('WARN< '+l)
+def LOG_INFO(l):
+    bIsVerbose = hasattr(__builtins__, 'app') and app.oArgs.loglevel <= 20-1
+    if bIsVerbose: print('INFO< '+l)
+def LOG_DEBUG(l):
+    bIsVerbose = hasattr(__builtins__, 'app') and app.oArgs.loglevel <= 10-1
+    if bIsVerbose: print('DBUG< '+l)
+def LOG_TRACE(l):
+    bIsVerbose = hasattr(__builtins__, 'app') and app.oArgs.loglevel < 10-1
+    pass # print('TRACE+ '+l)
+
+TIMER_TIMEOUT = 30.0
+bSTREAM_CALLBACK = False
+iFPS = 25
 
 class AV(common.tox_save.ToxAvSave):
 
     def __init__(self, toxav, settings):
         super().__init__(toxav)
+        self._toxav = toxav
         self._settings = settings
         self._running = True
+        s = settings
+        if 'video' not in s:
+            LOG.warn("AV.__init__ 'video' not in s" )
+            LOG.debug(f"AV.__init__ {s!r}" )
+        elif 'device' not in s['video']:
+            LOG.warn("AV.__init__ 'device' not in s.video" )
+            LOG.debug(f"AV.__init__ {s['video']!r}" )
 
         self._calls = {}  # dict: key - friend number, value - Call instance
 
@@ -25,17 +58,30 @@ class AV(common.tox_save.ToxAvSave):
         self._audio_running = False
         self._out_stream = None
 
-        self._audio_rate = 8000
         self._audio_channels = 1
         self._audio_duration = 60
-        self._audio_sample_count = self._audio_rate * self._audio_channels * self._audio_duration // 1000
+        self._audio_rate_pa = 48000
+        self._audio_rate_tox = 48000
+        self._audio_rate_pa = 48000
+        self._audio_krate_tox_audio = self._audio_rate_tox // 1000
+        self._audio_krate_tox_video = 5000
+        self._audio_sample_count_pa = self._audio_rate_pa * self._audio_channels * self._audio_duration // 1000
+        self._audio_sample_count_tox = self._audio_rate_tox * self._audio_channels * self._audio_duration // 1000
 
         self._video = None
         self._video_thread = None
-        self._video_running = False
+        self._video_running = None
 
-        self._video_width = 640
+        self._video_width = 320
-        self._video_height = 480
+        self._video_height = 240
+
+        # was iOutput = self._settings._args.audio['output']
+        iInput = self._settings['audio']['input']
+        self.lPaSampleratesI = ts.lSdSamplerates(iInput)
+        iOutput = self._settings['audio']['output']
+        self.lPaSampleratesO = ts.lSdSamplerates(iOutput)
+        global oPYA
+        oPYA = self._audio = pyaudio.PyAudio()
 
     def stop(self):
         self._running = False
@@ -51,28 +97,71 @@ class AV(common.tox_save.ToxAvSave):
 
     def __call__(self, friend_number, audio, video):
         """Call friend with specified number"""
-        self._toxav.call(friend_number, 32 if audio else 0, 5000 if video else 0)
+        if friend_number in self._calls:
+            LOG.warn(f"__call__ already has {friend_number}")
+            return
+        if self._audio_krate_tox_audio not in ts.lToxSampleratesK:
+            LOG.warn(f"__call__ {self._audio_krate_tox_audio} not in {ts.lToxSampleratesK}")
+
+        try:
+            self._toxav.call(friend_number,
+                             self._audio_krate_tox_audio if audio else 0,
+                             self._audio_krate_tox_video if video else 0)
+        except ArgumentError as e:
+            LOG.warn(f"_toxav.call already has {friend_number}")
+            return
         self._calls[friend_number] = Call(audio, video)
-        threading.Timer(30.0, lambda: self.finish_not_started_call(friend_number)).start()
+        threading.Timer(TIMER_TIMEOUT,
+                        lambda: self.finish_not_started_call(friend_number)).start()
 
     def accept_call(self, friend_number, audio_enabled, video_enabled):
+        # obsolete
+        return call_accept_call(self, friend_number, audio_enabled, video_enabled)
+
+    def call_accept_call(self, friend_number, audio_enabled, video_enabled):
+        LOG.debug(f"call_accept_call from {friend_number} {self._running}" +
+                  f"{audio_enabled} {video_enabled}")
+        # import pdb; pdb.set_trace() - gets into q Qt exec_ problem
+        # ts.trepan_handler()
+
+        if self._audio_krate_tox_audio not in ts.lToxSampleratesK:
+            LOG.warn(f"__call__ {self._audio_krate_tox_audio} not in {ts.lToxSampleratesK}")
         if self._running:
             self._calls[friend_number] = Call(audio_enabled, video_enabled)
-            self._toxav.answer(friend_number, 32 if audio_enabled else 0, 5000 if video_enabled else 0)
+            # audio_bit_rate: Audio bit rate in Kb/sec. Set this to 0 to disable audio sending.
+            # video_bit_rate: Video bit rate in Kb/sec. Set this to 0 to disable video sending.
+            try:
+                self._toxav.answer(friend_number,
+                                   self._audio_krate_tox_audio if audio_enabled else 0,
+                                   self._audio_krate_tox_video if video_enabled else 0)
+            except ArgumentError as e:
+                LOG.debug(f"AV accept_call error from {friend_number} {self._running}" +
+                          f"{e}")
+                raise
             if audio_enabled:
+                # may raise
                 self.start_audio_thread()
             if video_enabled:
+                # may raise
                 self.start_video_thread()
 
     def finish_call(self, friend_number, by_friend=False):
+        LOG.debug(f"finish_call  {friend_number}")
         if not by_friend:
             self._toxav.call_control(friend_number, TOXAV_CALL_CONTROL['CANCEL'])
         if friend_number in self._calls:
             del self._calls[friend_number]
+        try:
+            # AttributeError: 'int' object has no attribute 'out_audio'
             if not len(list(filter(lambda c: c.out_audio, self._calls))):
                 self.stop_audio_thread()
             if not len(list(filter(lambda c: c.out_video, self._calls))):
                 self.stop_video_thread()
+        except Exception as e:
+            LOG.error(f"finish_call FixMe:   {e}")
+            # dunno
+            self.stop_audio_thread()
+            self.stop_video_thread()
 
     def finish_not_started_call(self, friend_number):
         if friend_number in self:
@@ -84,6 +173,7 @@ class AV(common.tox_save.ToxAvSave):
         """
         New call state
         """
+        LOG.debug(f"toxav_call_state_cb {friend_number}")
         call = self._calls[friend_number]
         call.is_active = True
 
@@ -106,32 +196,85 @@ class AV(common.tox_save.ToxAvSave):
     def start_audio_thread(self):
         """
         Start audio sending
+        from a callback
         """
+        global oPYA
+        # was iInput = self._settings._args.audio['input']
+        iInput = self._settings['audio']['input']
         if self._audio_thread is not None:
+            LOG_WARN(f"start_audio_thread device={iInput}")
             return
+        LOG_DEBUG(f"start_audio_thread device={iInput}")
+        lPaSamplerates = ts.lSdSamplerates(iInput)
+        if not(len(lPaSamplerates)):
+            e = f"No supported sample rates for device: audio[input]={iInput!r}"
+            LOG_ERROR(f"start_audio_thread {e}")
+            #?? dunno - cancel call?
+            return
+        if not self._audio_rate_pa in lPaSamplerates:
+            LOG_WARN(f"{self._audio_rate_pa} not in {lPaSamplerates!r}")
+            if False:
+                self._audio_rate_pa = oPYA.get_device_info_by_index(iInput)['defaultSampleRate']
+            else:
+                LOG_WARN(f"Setting audio_rate to: {lPaSamplerates[0]}")
+                self._audio_rate_pa = lPaSamplerates[0]
 
-        self._audio_running = True
+        try:
+            LOG_DEBUG( f"start_audio_thread framerate: {self._audio_rate_pa}" \
+                     +f" device: {iInput}"
+                     +f" supported: {lPaSamplerates!r}")
+            if self._audio_rate_pa not in lPaSamplerates:
+                LOG_WARN(f"PAudio sampling rate was {self._audio_rate_pa} changed to {lPaSamplerates[0]}")
+                self._audio_rate_pa = lPaSamplerates[0]
 
-        self._audio = pyaudio.PyAudio()
+            if bSTREAM_CALLBACK:
-        self._audio_stream = self._audio.open(format=pyaudio.paInt16,
+                self._audio_stream = oPYA.open(format=pyaudio.paInt16,
-                                              rate=self._audio_rate,
+                                               rate=self._audio_rate_pa,
                                                channels=self._audio_channels,
                                                input=True,
-                                              input_device_index=self._settings.audio['input'],
+                                               input_device_index=iInput,
-                                              frames_per_buffer=self._audio_sample_count * 10)
+                                               frames_per_buffer=self._audio_sample_count_pa * 10,
+                                               stream_callback=self.send_audio_data)
+                self._audio_running = True
+                self._audio_stream.start_stream()
+                while self._audio_stream.is_active():
+                    sleep(0.1)
+                self._audio_stream.stop_stream()
+                self._audio_stream.close()
 
-        self._audio_thread = threading.Thread(target=self.send_audio)
+            else:
+                self._audio_stream = oPYA.open(format=pyaudio.paInt16,
+                                               rate=self._audio_rate_pa,
+                                               channels=self._audio_channels,
+                                               input=True,
+                                               input_device_index=iInput,
+                                               frames_per_buffer=self._audio_sample_count_pa * 10)
+                self._audio_running = True
+                self._audio_thread = BaseThread(target=self.send_audio,
+                                                      name='_audio_thread')
                 self._audio_thread.start()
 
+        except Exception as e:
+            LOG.error(f"Starting self._audio.open {e}")
+            LOG.debug(repr(dict(format=pyaudio.paInt16,
+                                rate=self._audio_rate_pa,
+                                channels=self._audio_channels,
+                                input=True,
+                                input_device_index=iInput,
+                                frames_per_buffer=self._audio_sample_count_pa * 10)))
+            # catcher in place in calls_manager? not if from a callback
+            # calls_manager._call.toxav_call_state_cb(friend_number, mask)
+            # raise RuntimeError(e)
+            return
+        else:
+            LOG_DEBUG(f"start_audio_thread {self._audio_stream!r}")
+
     def stop_audio_thread(self):
 
         if self._audio_thread is None:
             return
-
         self._audio_running = False
 
-        self._audio_thread.join()
-
         self._audio_thread = None
         self._audio_stream = None
         self._audio = None
@@ -144,21 +287,47 @@ class AV(common.tox_save.ToxAvSave):
     def start_video_thread(self):
         if self._video_thread is not None:
             return
+        s = self._settings
+        if 'video' not in s:
+            LOG.warn("AV.__init__ 'video' not in s" )
+            LOG.debug(f"start_video_thread {s!r}" )
+            raise RuntimeError("start_video_thread not 'video' in s)" )
+        elif 'device' not in s['video']:
+            LOG.error("start_video_thread not 'device' in s['video']" )
+            LOG.debug(f"start_video_thread {s['video']!r}" )
+            raise RuntimeError("start_video_thread not 'device' ins s['video']" )
+        self._video_width = s['video']['width']
+        self._video_height = s['video']['height']
 
-        self._video_running = True
+        if True or s['video']['device'] == -1:
-        self._video_width = s.video['width']
+            self._video = screen_sharing.DesktopGrabber(s['video']['x'],
-        self._video_height = s.video['height']
+                                                        s['video']['y'],
-
+                                                        s['video']['width'],
-        if s.video['device'] == -1:
+                                                        s['video']['height'])
-            self._video = screen_sharing.DesktopGrabber(self._settings.video['x'], self._settings.video['y'],
-                                                        self._settings.video['width'], self._settings.video['height'])
         else:
-            self._video = cv2.VideoCapture(self._settings.video['device'])
+            with ts.ignoreStdout():
-            self._video.set(cv2.CAP_PROP_FPS, 25)
+                import cv2
+            if s['video']['device'] == 0:
+                # webcam
+                self._video = cv2.VideoCapture(s['video']['device'], cv2.DSHOW)
+            else:
+                self._video = cv2.VideoCapture(s['video']['device'])
+            self._video.set(cv2.CAP_PROP_FPS, iFPS)
             self._video.set(cv2.CAP_PROP_FRAME_WIDTH, self._video_width)
             self._video.set(cv2.CAP_PROP_FRAME_HEIGHT, self._video_height)
+#            self._video.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'))
+        if self._video is None:
+            LOG.error("start_video_thread " \
+                     +f" device: {s['video']['device']}" \
+                     +f" supported: {s['video']['width']} {s['video']['height']}")
+            return
+        LOG.info("start_video_thread " \
+                 +f" device: {s['video']['device']}" \
+                 +f" supported: {s['video']['width']} {s['video']['height']}")
 
-        self._video_thread = threading.Thread(target=self.send_video)
+        self._video_running = True
+        self._video_thread = BaseThread(target=self.send_video,
+                                        name='_video_thread')
         self._video_thread.start()
 
     def stop_video_thread(self):
@@ -166,7 +335,17 @@ class AV(common.tox_save.ToxAvSave):
             return
 
         self._video_running = False
-        self._video_thread.join()
+        i = 0
+        while i < ts.iTHREAD_JOINS:
+            self._video_thread.join(ts.iTHREAD_TIMEOUT)
+            try:
+                if not self._video_thread.is_alive(): break
+            except:
+                # AttributeError: 'NoneType' object has no attribute 'join'
+                break
+            i = i + 1
+        else:
+            LOG.warn("self._video_thread.is_alive BLOCKED")
         self._video_thread = None
         self._video = None
 
@@ -180,58 +359,124 @@ class AV(common.tox_save.ToxAvSave):
         """
 
         if self._out_stream is None:
-            self._out_stream = self._audio.open(format=pyaudio.paInt16,
+            # was iOutput = self._settings._args.audio['output']
+            iOutput = self._settings['audio']['output']
+            if not rate in self.lPaSampleratesO:
+                LOG.warn(f"{rate} not in {self.lPaSampleratesO!r}")
+                if False:
+                    rate = oPYA.get_device_info_by_index(iOutput)['defaultSampleRate']
+                LOG.warn(f"Setting audio_rate to: {self.lPaSampleratesO[0]}")
+                rate = self.lPaSampleratesO[0]
+            try:
+                with ts.ignoreStderr():
+                    self._out_stream = oPYA.open(format=pyaudio.paInt16,
                                                  channels=channels_count,
                                                  rate=rate,
-                                                output_device_index=self._settings.audio['output'],
+                                                 output_device_index=iOutput,
                                                  output=True)
+            except Exception as e:
+                LOG.error(f"Error playing audio_chunk creating self._out_stream   {e}")
+                invoke_in_main_thread(util_ui.message_box,
+                                    str(e),
+                                    util_ui.tr("Error Chunking audio"))
+                # dunno
+                self.stop()
+                return
+
+        iOutput = self._settings['audio']['output']
+        LOG.debug(f"audio_chunk output_device_index={iOutput} rate={rate} channels={channels_count}")
         self._out_stream.write(samples)
 
     # -----------------------------------------------------------------------------------------------------------------
     # AV sending
     # -----------------------------------------------------------------------------------------------------------------
 
+    def send_audio_data(self, data, count, *largs, **kwargs):
+        pcm = data
+        # :param sampling_rate: Audio sampling rate used in this frame.
+        if self._toxav is None:
+            raise RuntimeError("_toxav not initialized")
+        if self._audio_rate_tox not in ts.lToxSamplerates:
+            LOG.warn(f"ToxAudio sampling rate was {self._audio_rate_tox} changed to {ts.lToxSamplerates[0]}")
+            self._audio_rate_tox = ts.lToxSamplerates[0]
+
+        for friend_num in self._calls:
+            if self._calls[friend_num].out_audio:
+                try:
+                    # app.av.calls ERROR Error send_audio:   One of the frame parameters was invalid. E.g. the resolution may be too small or too large, or the audio sampling rate may be unsupported
+                    self._toxav.audio_send_frame(friend_num,
+                                                 pcm,
+                                                 count,
+                                                 self._audio_channels,
+                                                 self._audio_rate_tox)
+                except Exception as e:
+                   LOG.error(f"Error send_audio audio_send_frame: {e}")
+                   LOG.debug(f"send_audio self._audio_rate_tox={self._audio_rate_tox} self._audio_channels={self._audio_channels}")
+                   invoke_in_main_thread(util_ui.message_box,
+                                    str(e),
+                                    util_ui.tr("Error send_audio audio_send_frame"))
+                   pass
+
     def send_audio(self):
         """
         This method sends audio to friends
         """
-
+        i=0
+        count = self._audio_sample_count_tox
+        LOG.debug(f"send_audio stream={self._audio_stream}")
         while self._audio_running:
             try:
-                pcm = self._audio_stream.read(self._audio_sample_count)
+                pcm = self._audio_stream.read(count, exception_on_overflow=False)
-                if pcm:
+                if not pcm:
-                    for friend_num in self._calls:
+                    sleep(0.1)
-                        if self._calls[friend_num].out_audio:
+                else:
-                            try:
+                    self.send_audio_data(pcm, count)
-                                self._toxav.audio_send_frame(friend_num, pcm, self._audio_sample_count,
-                                                             self._audio_channels, self._audio_rate)
             except:
                 pass
-            except:
+            i += 1
-                pass
+            LOG.debug(f"send_audio {i}")
-
+            sleep(0.01)
-            time.sleep(0.01)
 
     def send_video(self):
         """
         This method sends video to friends
         """
+        LOG.debug(f"send_video thread={threading.current_thread().name}"
+                  +f" self._video_running={self._video_running}"
+                  +f" device: {self._settings['video']['device']}" )
         while self._video_running:
             try:
                 result, frame = self._video.read()
-                if result:
+                if not result:
+                    LOG.warn(f"send_video video_send_frame _video.read result={result}")
+                    break
+                if frame is None:
+                    LOG.warn(f"send_video video_send_frame _video.read result={result} frame={frame}")
+                    continue
+                else:
+                    LOG.debug(f"send_video video_send_frame _video.read result={result}")
                     height, width, channels = frame.shape
+                    friends = []
                     for friend_num in self._calls:
                         if self._calls[friend_num].out_video:
+                            friends.append(friend_num)
+                    if len(friends) == 0:
+                        LOG.warn(f"send_video video_send_frame no friends")
+                    else:
+                        LOG.debug(f"send_video video_send_frame {friends}")
+                        friend_num = friends[0]
                         try:
                             y, u, v = self.convert_bgr_to_yuv(frame)
                             self._toxav.video_send_frame(friend_num, width, height, y, u, v)
-                            except:
+                        except Exception as e:
-                                pass
+                            LOG.debug(f"send_video video_send_frame ERROR {e}")
-            except:
                             pass
 
-            time.sleep(0.01)
+            except Exception as e:
+                LOG.error(f"send_video video_send_frame {e}")
+                pass
+
+            sleep( 1.0/iFPS)
 
     def convert_bgr_to_yuv(self, frame):
         """
@@ -264,11 +509,14 @@ class AV(common.tox_save.ToxAvSave):
         Y, U, V can be extracted using slices and joined in one list using itertools.chain.from_iterable()
         Function returns bytes(y), bytes(u), bytes(v), because it is required for ctypes
         """
+        with ts.ignoreStdout():
+            import cv2
         frame = cv2.cvtColor(frame, cv2.COLOR_BGR2YUV_I420)
 
         y = frame[:self._video_height, :]
         y = list(itertools.chain.from_iterable(y))
 
+        import numpy as np
         u = np.zeros((self._video_height // 2, self._video_width // 2), dtype=np.int)
         u[::2, :] = frame[self._video_height:self._video_height * 5 // 4, :self._video_width // 2]
         u[1::2, :] = frame[self._video_height:self._video_height * 5 // 4, self._video_width // 2:]

toxygen/av/calls_manager.py

@@ -1,25 +1,34 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+
+import sys
 import threading
-import cv2
+
 import av.calls
 from messenger.messages import *
 from ui import av_widgets
 import common.event as event
+import utils.ui as util_ui
 
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
 
 class CallsManager:
 
-    def __init__(self, toxav, settings, screen, contacts_manager):
+    def __init__(self, toxav, settings, main_screen, contacts_manager, app=None):
-        self._call = av.calls.AV(toxav, settings)  # object with data about calls
+        self._callav = av.calls.AV(toxav, settings)  # object with data about calls
+        self._call = self._callav
         self._call_widgets = {}  # dict of incoming call widgets
         self._incoming_calls = set()
         self._settings = settings
-        self._screen = screen
+        self._main_screen = main_screen
         self._contacts_manager = contacts_manager
         self._call_started_event = event.Event()  # friend_number, audio, video, is_outgoing
         self._call_finished_event = event.Event()  # friend_number, is_declined
+        self._app = app
 
     def set_toxav(self, toxav):
-        self._call.set_toxav(toxav)
+        self._callav.set_toxav(toxav)
 
     # -----------------------------------------------------------------------------------------------------------------
     # Events
@@ -44,26 +53,26 @@ class CallsManager:
         num = self._contacts_manager.get_active_number()
         if not self._contacts_manager.is_active_a_friend():
             return
-        if num not in self._call and self._contacts_manager.is_active_online():  # start call
+        if num not in self._callav and self._contacts_manager.is_active_online():  # start call
-            if not self._settings.audio['enabled']:
+            if not self._settings['audio']['enabled']:
                 return
-            self._call(num, audio, video)
+            self._callav(num, audio, video)
-            self._screen.active_call()
+            self._main_screen.active_call()
             self._call_started_event(num, audio, video, True)
-        elif num in self._call:  # finish or cancel call if you call with active friend
+        elif num in self._callav:  # finish or cancel call if you call with active friend
             self.stop_call(num, False)
 
     def incoming_call(self, audio, video, friend_number):
         """
         Incoming call from friend.
         """
-        if not self._settings.audio['enabled']:
+        LOG.debug(__name__ +f" incoming_call  {friend_number}")
-            return
+        # if not self._settings['audio']['enabled']: return
         friend = self._contacts_manager.get_friend_by_number(friend_number)
         self._call_started_event(friend_number, audio, video, False)
         self._incoming_calls.add(friend_number)
         if friend_number == self._contacts_manager.get_active_number():
-            self._screen.incoming_call()
+            self._main_screen.incoming_call()
         else:
             friend.actions = True
         text = util_ui.tr("Incoming video call") if video else util_ui.tr("Incoming audio call")
@@ -74,39 +83,81 @@ class CallsManager:
     def accept_call(self, friend_number, audio, video):
         """
         Accept incoming call with audio or video
+        Called from a thread
         """
-        self._call.accept_call(friend_number, audio, video)
+
-        self._screen.active_call()
+        LOG.debug(f"CM accept_call from {friend_number} {audio} {video}")
+        sys.stdout.flush()
+
+        try:
+            self._callav.call_accept_call(friend_number, audio, video)
+        except Exception as e:
+            LOG.error(f"accept_call _call.accept_call ERROR for {friend_number} {e}")
+            self._main_screen.call_finished()
+            if hasattr(self._main_screen, '_settings') and \
+              'audio' in self._main_screen._settings and \
+              'input' in self._main_screen._settings['audio']:
+                iInput = self._settings['audio']['input']
+                iOutput = self._settings['audio']['output']
+                iVideo = self._settings['video']['device']
+                LOG.debug(f"iInput={iInput} iOutput={iOutput} iVideo={iVideo}")
+            elif hasattr(self._main_screen, '_settings') and \
+              hasattr(self._main_screen._settings, 'audio') and \
+              'input' not in self._main_screen._settings['audio']:
+                LOG.warn(f"'audio' not in {self._main_screen._settings!r}")
+            elif hasattr(self._main_screen, '_settings') and \
+              hasattr(self._main_screen._settings, 'audio') and \
+              'input' not in self._main_screen._settings['audio']:
+                LOG.warn(f"'audio' not in {self._main_screen._settings!r}")
+            else:
+                LOG.warn(f"_settings not in self._main_screen")
+            util_ui.message_box(str(e),
+                            util_ui.tr('ERROR Accepting call from {friend_number}'))
+        else:
+            self._main_screen.active_call()
+
+        finally:
+            # does not terminate call - just the av_widget
             if friend_number in self._incoming_calls:
                 self._incoming_calls.remove(friend_number)
+            try:
+                self._call_widgets[friend_number].close()
                 del self._call_widgets[friend_number]
+            except:
+                # RuntimeError: wrapped C/C++ object of type IncomingCallWidget has been deleted
+
+                pass
+            LOG.debug(f" closed self._call_widgets[{friend_number}]")
 
     def stop_call(self, friend_number, by_friend):
         """
         Stop call with friend
         """
+        LOG.debug(__name__+f" stop_call {friend_number}")
         if friend_number in self._incoming_calls:
             self._incoming_calls.remove(friend_number)
             is_declined = True
         else:
             is_declined = False
-        self._screen.call_finished()
+        self._main_screen.call_finished()
-        is_video = self._call.is_video_call(friend_number)
+        self._callav.finish_call(friend_number, by_friend)  # finish or decline call
-        self._call.finish_call(friend_number, by_friend)  # finish or decline call
         if friend_number in self._call_widgets:
             self._call_widgets[friend_number].close()
             del self._call_widgets[friend_number]
 
         def destroy_window():
+            #??? FixMed
+            is_video = self._callav.is_video_call(friend_number)
             if is_video:
+                import cv2
                 cv2.destroyWindow(str(friend_number))
 
         threading.Timer(2.0, destroy_window).start()
         self._call_finished_event(friend_number, is_declined)
 
     def friend_exit(self, friend_number):
-        if friend_number in self._call:
+        if friend_number in self._callav:
-            self._call.finish_call(friend_number, True)
+            self._callav.finish_call(friend_number, True)
 
     # -----------------------------------------------------------------------------------------------------------------
     # Private methods

toxygen/av/screen_sharing.py

@@ -1,4 +1,3 @@
-import numpy as np
 from PyQt5 import QtWidgets
 
 
@@ -17,6 +16,7 @@ class DesktopGrabber:
         pixmap = self._screen.grabWindow(0, self._x, self._y, self._width, self._height)
         image = pixmap.toImage()
         s = image.bits().asstring(self._width * self._height * 4)
+        import numpy as np
         arr = np.fromstring(s, dtype=np.uint8).reshape((self._height, self._width, 4))
 
         return True, arr

toxygen/bootstrap/bootstrap.py

@@ -1,83 +1,48 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 import random
 import urllib.request
 from utils.util import *
-from PyQt5 import QtNetwork, QtCore
+from PyQt5 import QtNetwork
-import json
+from PyQt5 import QtCore
+try:
+    import certifi
+    from io import BytesIO
+except ImportError:
+    certifi = None
 
+from user_data.settings import get_user_config_path
+from wrapper_tests.support_testing import _get_nodes_path
+from wrapper_tests.support_http import download_url
+import wrapper_tests.support_testing as ts
 
-DEFAULT_NODES_COUNT = 4
+global LOG
+import logging
+LOG = logging.getLogger('app.'+'bootstrap')
 
-
+def download_nodes_list(settings, oArgs):
-class Node:
-
-    def __init__(self, node):
-        self._ip, self._port, self._tox_key = node['ipv4'], node['port'], node['public_key']
-        self._priority = random.randint(1, 1000000) if node['status_tcp'] and node['status_udp'] else 0
-
-    def get_priority(self):
-        return self._priority
-
-    priority = property(get_priority)
-
-    def get_data(self):
-        return self._ip, self._port, self._tox_key
-
-
-def generate_nodes(nodes_count=DEFAULT_NODES_COUNT):
-    with open(_get_nodes_path(), 'rt') as fl:
-        json_nodes = json.loads(fl.read())['nodes']
-    nodes = map(lambda json_node: Node(json_node), json_nodes)
-    nodes = filter(lambda n: n.priority > 0, nodes)
-    sorted_nodes = sorted(nodes, key=lambda x: x.priority)
-    if nodes_count is not None:
-        sorted_nodes = sorted_nodes[-DEFAULT_NODES_COUNT:]
-    for node in sorted_nodes:
-        yield node.get_data()
-
-
-def download_nodes_list(settings):
-    url = 'https://nodes.tox.chat/json'
     if not settings['download_nodes_list']:
-        return
+        return ''
+    if not ts.bAreWeConnected():
+        return ''        
+    url = settings['download_nodes_url']
+    path = _get_nodes_path(oArgs=oArgs)
+    # dont download blindly so we can edit the file and not block on startup
+    if os.path.isfile(path):
+        with open(path, 'rt') as fl:
+            result = fl.read()
+            return result
+    LOG.debug("downloading list of nodes")
+    result = download_url(url, settings._app._settings)
+    if not result:
+        LOG.warn("failed downloading list of nodes")
+        return ''
+    LOG.info("downloaded list of nodes")
+    _save_nodes(result, settings._app)
+    return result
 
-    if not settings['proxy_type']:  # no proxy
+def _save_nodes(nodes, app):
-        try:
-            req = urllib.request.Request(url)
-            req.add_header('Content-Type', 'application/json')
-            response = urllib.request.urlopen(req)
-            result = response.read()
-            _save_nodes(result)
-        except Exception as ex:
-            log('TOX nodes loading error: ' + str(ex))
-    else:  # proxy
-        netman = QtNetwork.QNetworkAccessManager()
-        proxy = QtNetwork.QNetworkProxy()
-        proxy.setType(
-            QtNetwork.QNetworkProxy.Socks5Proxy if settings['proxy_type'] == 2 else QtNetwork.QNetworkProxy.HttpProxy)
-        proxy.setHostName(settings['proxy_host'])
-        proxy.setPort(settings['proxy_port'])
-        netman.setProxy(proxy)
-        try:
-            request = QtNetwork.QNetworkRequest()
-            request.setUrl(QtCore.QUrl(url))
-            reply = netman.get(request)
-
-            while not reply.isFinished():
-                QtCore.QThread.msleep(1)
-                QtCore.QCoreApplication.processEvents()
-            data = bytes(reply.readAll().data())
-            _save_nodes(data)
-        except Exception as ex:
-            log('TOX nodes loading error: ' + str(ex))
-
-
-def _get_nodes_path():
-    return join_path(curr_directory(__file__), 'nodes.json')
-
-
-def _save_nodes(nodes):
     if not nodes:
         return
-    print('Saving nodes...')
+    with open(_get_nodes_path(oArgs=app._args), 'wb') as fl:
-    with open(_get_nodes_path(), 'wb') as fl:
+        LOG.info("Saving nodes to " +_get_nodes_path())
         fl.write(nodes)

toxygen/contacts/basecontact.py

@@ -1,3 +1,4 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 from user_data.settings import *
 from PyQt5 import QtCore, QtGui
 from wrapper.toxcore_enums_and_consts import TOX_PUBLIC_KEY_SIZE
@@ -14,17 +15,20 @@ class BaseContact:
     Base class for all contacts.
     """
 
-    def __init__(self, profile_manager, name, status_message, widget, tox_id):
+    def __init__(self, profile_manager, name, status_message, widget, tox_id, kind=''):
         """
         :param name: name, example: 'Toxygen user'
         :param status_message: status message, example: 'Toxing on Toxygen'
         :param widget: ContactItem instance
         :param tox_id: tox id of contact
+        :param kind: one of ['bot', 'friend', 'group', 'invite', 'grouppeer', '']
         """
         self._profile_manager = profile_manager
         self._name, self._status_message = name, status_message
+        self._kind = kind
         self._status, self._widget = None, widget
         self._tox_id = tox_id
+
         self._name_changed_event = event.Event()
         self._status_message_changed_event = event.Event()
         self._status_changed_event = event.Event()
@@ -168,6 +172,8 @@ class BaseContact:
     def init_widget(self):
         self._widget.name.setText(self._name)
         self._widget.status_message.setText(self._status_message)
+        if hasattr(self._widget, 'kind'):
+            self._widget.kind.setText(self._kind)
         self._widget.connection_status.update(self._status)
         self.load_avatar()
 

toxygen/contacts/contact.py

@@ -1,10 +1,17 @@
-from history.database import *
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+from history.database import TIMEOUT, \
+    SAVE_MESSAGES, MESSAGE_AUTHOR
+
 from contacts import basecontact, common
 from messenger.messages import *
 from contacts.contact_menu import *
 from file_transfers import file_transfers as ft
 import re
 
+# LOG=util.log
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
 
 class Contact(basecontact.BaseContact):
     """
@@ -139,7 +146,7 @@ class Contact(basecontact.BaseContact):
                                             and m.tox_message_id == tox_message_id, self._corr))[0]
             message.mark_as_sent()
         except Exception as ex:
-            util.log('Mark as sent ex: ' + str(ex))
+            LOG.error(f"Mark as sent:  {ex!s}")
 
     # -----------------------------------------------------------------------------------------------------------------
     # Message deletion

toxygen/contacts/contact_menu.py

@@ -1,6 +1,12 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 from PyQt5 import QtWidgets
-import utils.ui as util_ui
 
+import utils.ui as util_ui
+from wrapper.toxcore_enums_and_consts import *
+
+global LOG
+import logging
+LOG = logging.getLogger('app')
 
 # -----------------------------------------------------------------------------------------------------------------
 # Builder
@@ -99,8 +105,8 @@ class BaseContactMenuGenerator:
         (copy_menu_builder
          .with_name(util_ui.tr('Copy'))
          .with_action(util_ui.tr('Name'), lambda: main_screen.copy_text(self._contact.name))
-         .with_action(util_ui.tr('Status message'), lambda: main_screen.copy_text(self._contact.status_message))
+         .with_action(util_ui.tr("Status message"), lambda: main_screen.copy_text(self._contact.status_message))
-         .with_action(util_ui.tr('Public key'), lambda: main_screen.copy_text(self._contact.tox_id))
+         .with_action(util_ui.tr("Public key"), lambda: main_screen.copy_text(self._contact.tox_id))
          )
 
         return copy_menu_builder
@@ -108,11 +114,11 @@ class BaseContactMenuGenerator:
     def _generate_history_menu_builder(self, history_loader, main_screen):
         history_menu_builder = ContactMenuBuilder()
         (history_menu_builder
-         .with_name(util_ui.tr('Chat history'))
+         .with_name(util_ui.tr("Chat history"))
-         .with_action(util_ui.tr('Clear history'), lambda: history_loader.clear_history(self._contact)
+         .with_action(util_ui.tr("Clear history"), lambda: history_loader.clear_history(self._contact)
                                                            or main_screen.messages.clear())
-         .with_action(util_ui.tr('Export as text'), lambda: history_loader.export_history(self._contact))
+         .with_action(util_ui.tr("Export as text"), lambda: history_loader.export_history(self._contact))
-         .with_action(util_ui.tr('Export as HTML'), lambda: history_loader.export_history(self._contact, False))
+         .with_action(util_ui.tr("Export as HTML"), lambda: history_loader.export_history(self._contact, False))
          )
 
         return history_menu_builder
@@ -127,16 +133,16 @@ class FriendMenuGenerator(BaseContactMenuGenerator):
         groups_menu_builder = self._generate_groups_menu(contacts_manager, groups_service)
 
         allowed = self._contact.tox_id in settings['auto_accept_from_friends']
-        auto = util_ui.tr('Disallow auto accept') if allowed else util_ui.tr('Allow auto accept')
+        auto = util_ui.tr("Disallow auto accept") if allowed else util_ui.tr('Allow auto accept')
 
         builder = ContactMenuBuilder()
         menu = (builder
-                .with_action(util_ui.tr('Set alias'), lambda: main_screen.set_alias(number))
+                .with_action(util_ui.tr("Set alias"), lambda: main_screen.set_alias(number))
                 .with_submenu(history_menu_builder)
                 .with_submenu(copy_menu_builder)
                 .with_action(auto, lambda: main_screen.auto_accept(number, not allowed))
-                .with_action(util_ui.tr('Remove friend'), lambda: main_screen.remove_friend(number))
+                .with_action(util_ui.tr("Remove friend"), lambda: main_screen.remove_friend(number))
-                .with_action(util_ui.tr('Block friend'), lambda: main_screen.block_friend(number))
+                .with_action(util_ui.tr("Block friend"), lambda: main_screen.block_friend(number))
                 .with_action(util_ui.tr('Notes'), lambda: main_screen.show_note(self._contact))
                 .with_optional_submenu(plugins_menu_builder)
                 .with_optional_submenu(groups_menu_builder)
@@ -165,11 +171,13 @@ class FriendMenuGenerator(BaseContactMenuGenerator):
 
     def _generate_groups_menu(self, contacts_manager, groups_service):
         chats = contacts_manager.get_group_chats()
+        LOG.debug(f"_generate_groups_menu len(chats)={len(chats)} or self._contact.status={self._contact.status}")
         if not len(chats) or self._contact.status is None:
-            return None
+            #? return None
+            pass
         groups_menu_builder = ContactMenuBuilder()
         (groups_menu_builder
-         .with_name(util_ui.tr('Invite to group'))
+         .with_name(util_ui.tr("Invite to group"))
          .with_actions([(g.name, lambda: groups_service.invite_friend(self._contact.number, g.number)) for g in chats])
          )
 
@@ -184,26 +192,26 @@ class GroupMenuGenerator(BaseContactMenuGenerator):
 
         builder = ContactMenuBuilder()
         menu = (builder
-                .with_action(util_ui.tr('Set alias'), lambda: main_screen.set_alias(number))
+                .with_action(util_ui.tr("Set alias"), lambda: main_screen.set_alias(number))
                 .with_submenu(copy_menu_builder)
                 .with_submenu(history_menu_builder)
-                .with_optional_action(util_ui.tr('Manage group'),
+                .with_optional_action(util_ui.tr("Manage group"),
                                       lambda: groups_service.show_group_management_screen(self._contact),
                                       self._contact.is_self_founder())
-                .with_optional_action(util_ui.tr('Group settings'),
+                .with_optional_action(util_ui.tr("Group settings"),
                                       lambda: groups_service.show_group_settings_screen(self._contact),
                                       not self._contact.is_self_founder())
-                .with_optional_action(util_ui.tr('Set topic'),
+                .with_optional_action(util_ui.tr("Set topic"),
                                       lambda: groups_service.set_group_topic(self._contact),
                                       self._contact.is_self_moderator_or_founder())
-                .with_action(util_ui.tr('Bans list'),
+#                .with_action(util_ui.tr("Bans list"),
-                             lambda: groups_service.show_bans_list(self._contact))
+#                             lambda: groups_service.show_bans_list(self._contact))
-                .with_action(util_ui.tr('Reconnect to group'),
+                .with_action(util_ui.tr("Reconnect to group"),
                              lambda: groups_service.reconnect_to_group(self._contact.number))
-                .with_optional_action(util_ui.tr('Disconnect from group'),
+                .with_optional_action(util_ui.tr("Disconnect from group"),
                                       lambda: groups_service.disconnect_from_group(self._contact.number),
                                       self._contact.status is not None)
-                .with_action(util_ui.tr('Leave group'), lambda: groups_service.leave_group(self._contact.number))
+                .with_action(util_ui.tr("Leave group"), lambda: groups_service.leave_group(self._contact.number))
                 .with_action(util_ui.tr('Notes'), lambda: main_screen.show_note(self._contact))
                 ).build()
 
@@ -218,10 +226,10 @@ class GroupPeerMenuGenerator(BaseContactMenuGenerator):
 
         builder = ContactMenuBuilder()
         menu = (builder
-                .with_action(util_ui.tr('Set alias'), lambda: main_screen.set_alias(number))
+                .with_action(util_ui.tr("Set alias"), lambda: main_screen.set_alias(number))
                 .with_submenu(copy_menu_builder)
                 .with_submenu(history_menu_builder)
-                .with_action(util_ui.tr('Quit chat'),
+                .with_action(util_ui.tr("Quit chat"),
                              lambda: contacts_manager.remove_group_peer(self._contact))
                 .with_action(util_ui.tr('Notes'), lambda: main_screen.show_note(self._contact))
                 ).build()

toxygen/contacts/contact_provider.py

@@ -1,5 +1,11 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+
 import common.tox_save as tox_save
 
+global LOG
+import logging
+LOG = logging.getLogger(__name__)
+
 
 class ContactProvider(tox_save.ToxSave):
 
@@ -15,8 +21,10 @@ class ContactProvider(tox_save.ToxSave):
     # -----------------------------------------------------------------------------------------------------------------
 
     def get_friend_by_number(self, friend_number):
+        try:
             public_key = self._tox.friend_get_public_key(friend_number)
-
+        except Exception as e:
+            return None
         return self.get_friend_by_public_key(public_key)
 
     def get_friend_by_public_key(self, public_key):
@@ -29,7 +37,10 @@ class ContactProvider(tox_save.ToxSave):
         return friend
 
     def get_all_friends(self):
+        try:
             friend_numbers = self._tox.self_get_friend_list()
+        except Exception as e:
+            return None
         friends = map(lambda n: self.get_friend_by_number(n), friend_numbers)
 
         return list(friends)
@@ -39,15 +50,31 @@ class ContactProvider(tox_save.ToxSave):
     # -----------------------------------------------------------------------------------------------------------------
 
     def get_all_groups(self):
+        try:
             group_numbers = range(self._tox.group_get_number_groups())
+        except Exception as e:
+            return None
         groups = map(lambda n: self.get_group_by_number(n), group_numbers)
 
         return list(groups)
 
     def get_group_by_number(self, group_number):
+        try:
+            if True:
+                # original code
                 public_key = self._tox.group_get_chat_id(group_number)
-
+#                LOG.info(f"group_get_chat_id {group_number} {public_key}")
                 return self.get_group_by_public_key(public_key)
+            else:
+                # guessing
+                chat_id = self._tox.group_get_chat_id(group_number)
+#                LOG.info(f"group_get_chat_id {group_number} {chat_id}")
+                group = self.get_contact_by_tox_id(chat_id)
+                return group
+        except Exception as e:
+            LOG.warn(f"group_get_chat_id {group_number} {e}")
+            return None
+
 
     def get_group_by_public_key(self, public_key):
         group = self._get_contact_from_cache(public_key)
@@ -67,7 +94,7 @@ class ContactProvider(tox_save.ToxSave):
 
     def get_group_peer_by_id(self, group, peer_id):
         peer = group.get_peer_by_id(peer_id)
-
+        if peer:
             return self._get_group_peer(group, peer)
 
     def get_group_peer_by_public_key(self, group, public_key):

toxygen/contacts/contacts_manager.py

@@ -1,9 +1,42 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+
+import traceback
+
 from contacts.friend import Friend
 from contacts.group_chat import GroupChat
 from messenger.messages import *
 from common.tox_save import ToxSave
 from contacts.group_peer_contact import GroupPeerContact
+from groups.group_peer import GroupChatPeer
 
+# LOG=util.log
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
+
+def LOG_ERROR(l): print('ERROR_: '+l)
+def LOG_WARN(l): print('WARN_: '+l)
+def LOG_INFO(l): print('INFO_: '+l)
+def LOG_DEBUG(l): print('DEBUG_: '+l)
+def LOG_TRACE(l): pass # print('TRACE+ '+l)
+
+
+UINT32_MAX = 2 ** 32 -1
+
+def set_contact_kind(contact):
+    bInvite = len(contact.name) == TOX_PUBLIC_KEY_SIZE * 2 and \
+      contact.status_message == ''
+    bBot = not bInvite and contact.name.lower().endswith(' bot')
+    if type(contact) == Friend and bInvite:
+        contact._kind = 'invite'
+    elif type(contact) == Friend and bBot:
+        contact._kind = 'bot'
+    elif type(contact) == Friend:
+        contact._kind = 'friend'
+    elif type(contact) == GroupChat:
+        contact._kind = 'group'
+    elif type(contact) == GroupChatPeer:
+        contact._kind = 'grouppeer'
 
 class ContactsManager(ToxSave):
     """
@@ -15,6 +48,7 @@ class ContactsManager(ToxSave):
         super().__init__(tox)
         self._settings = settings
         self._screen = screen
+        self._ms = screen
         self._profile_manager = profile_manager
         self._contact_provider = contact_provider
         self._tox_dns = tox_dns
@@ -28,6 +62,11 @@ class ContactsManager(ToxSave):
         self._history = history
         self._load_contacts()
 
+    def _log(self, s):
+        try:
+            self._ms._log(s)
+        except: pass
+
     def get_contact(self, num):
         if num < 0 or num >= len(self._contacts):
             return None
@@ -53,6 +92,17 @@ class ContactsManager(ToxSave):
         return self.get_curr_contact().number == group_number
 
     def is_contact_active(self, contact):
+        if not self._active_contact:
+#            LOG.debug("No self._active_contact")
+            return False
+        if self._active_contact not in self._contacts:
+            LOG.warn(f"_active_contact={self._active_contact} not in contacts len={len(self._contacts)}")
+            return False
+        if not self._contacts[self._active_contact]:
+            LOG.debug(f"{self._contacts[self._active_contact]}  {contact.tox_id}")
+            return False
+
+        LOG.debug(f"{self._contacts[self._active_contact].tox_id} == {contact.tox_id}")
         return self._contacts[self._active_contact].tox_id == contact.tox_id
 
     # -----------------------------------------------------------------------------------------------------------------
@@ -100,6 +150,7 @@ class ContactsManager(ToxSave):
                     current_contact.curr_text = self._screen.messageEdit.toPlainText()
                 except:
                     pass
+            # IndexError: list index out of range
             contact = self._contacts[value]
             self._subscribe_to_events(contact)
             contact.remove_invalid_unsent_files()
@@ -129,9 +180,9 @@ class ContactsManager(ToxSave):
             self._set_current_contact_data(contact)
             self._active_contact_changed(contact)
         except Exception as ex:  # no friend found. ignore
-            util.log('Friend value: ' + str(value))
+            LOG.warn(f"no friend found. Friend value:  {value!s}")
-            util.log('Error in set active: ' + str(ex))
+            LOG.error('in set active: ' + str(ex))
-            raise
+            # gulp raise
 
     active_contact = property(get_active, set_active)
 
@@ -161,13 +212,17 @@ class ContactsManager(ToxSave):
         """
         Filtration of friends list
         :param sorting: 0 - no sorting, 1 - online only, 2 - online first, 3 - by name,
-        4 - online and by name, 5 - online first and by name
+        4 - online and by name, 5 - online first and by name, 6 kind
         :param filter_str: show contacts which name contains this substring
         """
         filter_str = filter_str.lower()
         current_contact = self.get_curr_contact()
 
-        if sorting > 5 or sorting < 0:
+        for index, contact in enumerate(self._contacts):
+            if not contact._kind:
+                set_contact_kind(contact)
+
+        if sorting > 6 or sorting < 0:
             sorting = 0
 
         if sorting in (1, 2, 4, 5):  # online first
@@ -183,14 +238,23 @@ class ContactsManager(ToxSave):
             part2 = sorted(part2, key=key_lambda)
             self._contacts = part1 + part2
         elif sorting == 0:
+            # AttributeError: 'NoneType' object has no attribute 'number'
+            for (i, contact) in enumerate(self._contacts):
+                if contact is None or not hasattr(contact, 'number'):
+                    LOG.error(f"Contact {i} is None or not hasattr 'number'")
+                    del self._contacts[i]
+                    continue
             contacts = sorted(self._contacts, key=lambda c: c.number)
             friends = filter(lambda c: type(c) is Friend, contacts)
             groups = filter(lambda c: type(c) is GroupChat, contacts)
             group_peers = filter(lambda c: type(c) is GroupPeerContact, contacts)
             self._contacts = list(friends) + list(groups) + list(group_peers)
+        elif sorting == 6:
+            self._contacts = sorted(self._contacts, key=lambda x: x._kind)
         else:
             self._contacts = sorted(self._contacts, key=lambda x: x.name.lower())
 
+
         # change item widgets
         for index, contact in enumerate(self._contacts):
             list_item = self._screen.friends_list.item(index)
@@ -235,10 +299,15 @@ class ContactsManager(ToxSave):
     def get_or_create_group_peer_contact(self, group_number, peer_id):
         group = self.get_group_by_number(group_number)
         peer = group.get_peer_by_id(peer_id)
+        if peer: # broken?
+            if not hasattr(peer, 'public_key') or not peer.public_key:
+                LOG.error(f'no peer public_key ' + repr(dir(peer)))
+            else:
                 if not self.check_if_contact_exists(peer.public_key):
                     self.add_group_peer(group, peer)
-
                 return self.get_contact_by_tox_id(peer.public_key)
+        else:
+            LOG.warn(f'no peer group_number={group_number} peer_id={peer_id}')
 
     def check_if_contact_exists(self, tox_id):
         return any(filter(lambda c: c.tox_id == tox_id, self._contacts))
@@ -315,7 +384,7 @@ class ContactsManager(ToxSave):
         Block user with specified tox id (or public key) - delete from friends list and ignore friend requests
         """
         tox_id = tox_id[:TOX_PUBLIC_KEY_SIZE * 2]
-        if tox_id == self._tox.self_get_address[:TOX_PUBLIC_KEY_SIZE * 2]:
+        if tox_id == self._tox.self_get_address()[:TOX_PUBLIC_KEY_SIZE * 2]:
             return
         if tox_id not in self._settings['blocked']:
             self._settings['blocked'].append(tox_id)
@@ -347,11 +416,17 @@ class ContactsManager(ToxSave):
         return list(filter(lambda c: type(c) is GroupChat, self._contacts))
 
     def add_group(self, group_number):
-        group = self._contact_provider.get_group_by_number(group_number)
         index = len(self._contacts)
+        group = self._contact_provider.get_group_by_number(group_number)
+        if not group:
+            LOG.warn(f"CM.add_group: NO group {group_number}")
+        else:
+            LOG.info(f"CM.add_group: Adding group {group._name}")
             self._contacts.append(group)
-        group.reset_avatar(self._settings['identicons'])
+            LOG.info(f"contacts_manager.add_group: saving profile")
             self._save_profile()
+            group.reset_avatar(self._settings['identicons'])
+            LOG.info(f"contacts_manager.add_group: setting active")
             self.set_active(index)
             self.update_filtration()
 
@@ -369,12 +444,14 @@ class ContactsManager(ToxSave):
         contact = self._contact_provider.get_group_peer_by_id(group, peer.id)
         if self.check_if_contact_exists(contact.tox_id):
             return
+        contact._kind = 'grouppeer'
         self._contacts.append(contact)
         contact.reset_avatar(self._settings['identicons'])
         self._save_profile()
 
     def remove_group_peer_by_id(self, group, peer_id):
         peer = group.get_peer_by_id(peer_id)
+        if peer: # broken
             if not self.check_if_contact_exists(peer.public_key):
                 return
             contact = self.get_contact_by_tox_id(peer.public_key)
@@ -382,6 +459,7 @@ class ContactsManager(ToxSave):
 
     def remove_group_peer(self, group_peer_contact):
         contact = self.get_contact_by_tox_id(group_peer_contact.tox_id)
+        if contact:
             self._cleanup_contact_data(contact)
             num = self._contacts.index(contact)
             self._delete_contact(num)
@@ -406,34 +484,45 @@ class ContactsManager(ToxSave):
     # Friend requests
     # -----------------------------------------------------------------------------------------------------------------
 
-    def send_friend_request(self, tox_id, message):
+    def send_friend_request(self, sToxPkOrId, message):
         """
         Function tries to send request to contact with specified id
-        :param tox_id: id of new contact or tox dns 4 value
+        :param sToxPkOrId: id of new contact or tox dns 4 value
         :param message: additional message
         :return: True on success else error string
         """
+        retval = ''
         try:
             message = message or 'Hello! Add me to your contact list please'
-            if '@' in tox_id:  # value like groupbot@toxme.io
+            if len(sToxPkOrId) == TOX_PUBLIC_KEY_SIZE * 2:  # public key
-                tox_id = self._tox_dns.lookup(tox_id)
+                self.add_friend(sToxPkOrId)
-                if tox_id is None:
+                title = 'Friend added'
-                    raise Exception('TOX DNS lookup failed')
+                text = 'Friend added without sending friend request'
-            if len(tox_id) == TOX_PUBLIC_KEY_SIZE * 2:  # public key
-                self.add_friend(tox_id)
-                title = util_ui.tr('Friend added')
-                text = util_ui.tr('Friend added without sending friend request')
-                util_ui.message_box(text, title)
             else:
-                self._tox.friend_add(tox_id, message.encode('utf-8'))
+                num = self._tox.friend_add(sToxPkOrId, message.encode('utf-8'))
-                tox_id = tox_id[:TOX_PUBLIC_KEY_SIZE * 2]
+                if num < UINT32_MAX:
-                self._add_friend(tox_id)
+                    tox_pk = sToxPkOrId[:TOX_PUBLIC_KEY_SIZE * 2]
+                    self._add_friend(tox_pk)
                     self.update_filtration()
+                    title = 'Friend added'
+                    text = 'Friend added by sending friend request'
                     self.save_profile()
-            return True
+                    retval = True
+                else:
+                    title = 'Friend failed'
+                    text = 'Friend failed sending friend request'
+                    retval = text
+
         except Exception as ex:  # wrong data
-            util.log('Friend request failed with ' + str(ex))
+            title = 'Friend add exception'
-            return str(ex)
+            text = 'Friend request exception with ' + str(ex)
+            self._log(text)
+            LOG.error(traceback.format_exc())
+            retval = str(ex)
+        title = util_ui.tr(title)
+        text = util_ui.tr(text)
+        util_ui.message_box(text, title)
+        return retval
 
     def process_friend_request(self, tox_id, message):
         """
@@ -451,7 +540,7 @@ class ContactsManager(ToxSave):
                 data = self._tox.get_savedata()
                 self._profile_manager.save_profile(data)
         except Exception as ex:  # something is wrong
-            util.log('Accept friend request failed! ' + str(ex))
+            LOG.error('Accept friend request failed! ' + str(ex))
 
     def can_send_typing_notification(self):
         return self._settings['typing_notifications'] and not self.is_active_a_group_chat_peer()
@@ -467,9 +556,17 @@ class ContactsManager(ToxSave):
 
     def update_groups_numbers(self):
         groups = self._contact_provider.get_all_groups()
+        LOG.info(f"update_groups_numbers len(groups)={len(groups)}")
+        # Thread 76 "ToxIterateThrea" received signal SIGSEGV, Segmentation fault.
         for i in range(len(groups)):
             chat_id = self._tox.group_get_chat_id(i)
+            if not chat_id:
+                LOG.warn(f"update_groups_numbers {i} chat_id")
+                continue
             group = self.get_contact_by_tox_id(chat_id)
+            if not group:
+                LOG.warn(f"update_groups_numbers {i} group")
+                continue
             group.number = i
         self.update_filtration()
 
@@ -487,7 +584,13 @@ class ContactsManager(ToxSave):
         self._load_groups()
         if len(self._contacts):
             self.set_active(0)
-        for contact in filter(lambda c: not c.has_avatar(), self._contacts):
+        # filter(lambda c: not c.has_avatar(), self._contacts)
+        for (i, contact) in enumerate(self._contacts):
+            if not contact:
+                LOG.warn("_load_contacts NULL contact {i}")
+                del self._contacts[i]
+                continue
+            if contact.has_avatar(): continue
             contact.reset_avatar(self._settings['identicons'])
         self.update_filtration()
 

toxygen/contacts/friend_factory.py

@@ -13,21 +13,21 @@ class FriendFactory(ToxSave):
 
     def create_friend_by_public_key(self, public_key):
         friend_number = self._tox.friend_by_public_key(public_key)
-
         return self.create_friend_by_number(friend_number)
 
     def create_friend_by_number(self, friend_number):
         aliases = self._settings['friends_aliases']
-        tox_id = self._tox.friend_get_public_key(friend_number)
+        sToxPk = self._tox.friend_get_public_key(friend_number)
+        assert sToxPk, sToxPk
         try:
-            alias = list(filter(lambda x: x[0] == tox_id, aliases))[0][1]
+            alias = list(filter(lambda x: x[0] == sToxPk, aliases))[0][1]
         except:
             alias = ''
         item = self._create_friend_item()
-        name = alias or self._tox.friend_get_name(friend_number) or tox_id
+        name = alias or self._tox.friend_get_name(friend_number) or sToxPk
         status_message = self._tox.friend_get_status_message(friend_number)
-        message_getter = self._db.messages_getter(tox_id)
+        message_getter = self._db.messages_getter(sToxPk)
-        friend = Friend(self._profile_manager, message_getter, friend_number, name, status_message, item, tox_id)
+        friend = Friend(self._profile_manager, message_getter, friend_number, name, status_message, item, sToxPk)
         friend.set_alias(alias)
 
         return friend

toxygen/contacts/group_chat.py

@@ -1,3 +1,5 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+
 from contacts import contact
 from contacts.contact_menu import GroupMenuGenerator
 import utils.util as util
@@ -6,6 +8,14 @@ from wrapper import toxcore_enums_and_consts as constants
 from common.tox_save import ToxSave
 from groups.group_ban import GroupBan
 
+global LOG
+import logging
+LOG = logging.getLogger(__name__)
+def LOG_ERROR(l): print('ERROR_: '+l)
+def LOG_WARN(l): print('WARN_: '+l)
+def LOG_INFO(l): print('INFO_: '+l)
+def LOG_DEBUG(l): print('DEBUG_: '+l)
+def LOG_TRACE(l): pass # print('TRACE+ '+l)
 
 class GroupChat(contact.Contact, ToxSave):
 
@@ -73,6 +83,12 @@ class GroupChat(contact.Contact, ToxSave):
         return self.get_self_role() == constants.TOX_GROUP_ROLE['FOUNDER']
 
     def add_peer(self, peer_id, is_current_user=False):
+        "called from callbacks"
+        if peer_id >  self._peers_limit:
+            LOG_WARN(f"add_peer id={peer_id} > {self._peers_limit}")
+            return
+
+        LOG_TRACE(f"add_peer id={peer_id}")
         peer = GroupChatPeer(peer_id,
                              self._tox.group_peer_get_name(self._number, peer_id),
                              self._tox.group_peer_get_status(self._number, peer_id),
@@ -86,25 +102,40 @@ class GroupChat(contact.Contact, ToxSave):
             self.remove_all_peers_except_self()
         else:
             peer = self.get_peer_by_id(peer_id)
+            if peer: # broken
                 self._peers.remove(peer)
+            else:
+                LOG_WARN(f"remove_peer empty peers for {peer_id}")
 
     def get_peer_by_id(self, peer_id):
         peers = list(filter(lambda p: p.id == peer_id, self._peers))
-
+        if peers:
             return peers[0]
+        else:
+            LOG_WARN(f"get_peer_by_id empty peers for {peer_id}")
+            return []
 
     def get_peer_by_public_key(self, public_key):
         peers = list(filter(lambda p: p.public_key == public_key, self._peers))
-
+        # DEBUGc: group_moderation #0 mod_id=4294967295 event_type=3
+        # WARN_: get_peer_by_id empty peers for 4294967295
+        if peers:
             return peers[0]
+        else:
+            LOG_WARN(f"get_peer_by_public_key empty peers for {public_key}")
+            return []
 
     def remove_all_peers_except_self(self):
         self._peers = self._peers[:1]
 
     def get_peers_names(self):
         peers_names = map(lambda p: p.name, self._peers)
-
+        if peers_names: # broken
             return list(peers_names)
+        else:
+            LOG_WARN(f"get_peers_names empty peers")
+            #? broken
+            return []
 
     def get_peers(self):
         return self._peers[:]
@@ -112,16 +143,17 @@ class GroupChat(contact.Contact, ToxSave):
     peers = property(get_peers)
 
     def get_bans(self):
-        ban_ids = self._tox.group_ban_get_list(self._number)
+        return []
-        bans = []
+#        ban_ids = self._tox.group_ban_get_list(self._number)
-        for ban_id in ban_ids:
+#        bans = []
-            ban = GroupBan(ban_id,
+#        for ban_id in ban_ids:
-                           self._tox.group_ban_get_target(self._number, ban_id),
+#            ban = GroupBan(ban_id,
-                           self._tox.group_ban_get_time_set(self._number, ban_id))
+#                           self._tox.group_ban_get_target(self._number, ban_id),
-            bans.append(ban)
+#                           self._tox.group_ban_get_time_set(self._number, ban_id))
-
+#            bans.append(ban)
-        return bans
+#
-
+#        return bans
+#
     bans = property(get_bans)
 
     # -----------------------------------------------------------------------------------------------------------------

toxygen/contacts/group_factory.py

@@ -1,7 +1,12 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+
 from contacts.group_chat import GroupChat
 from common.tox_save import ToxSave
 import wrapper.toxcore_enums_and_consts as constants
 
+global LOG
+import logging
+LOG = logging.getLogger(__name__)
 
 class GroupFactory(ToxSave):
 
@@ -18,6 +23,7 @@ class GroupFactory(ToxSave):
         return self.create_group_by_number(group_number)
 
     def create_group_by_number(self, group_number):
+        LOG.info(f"create_group_by_number {group_number}")
         aliases = self._settings['friends_aliases']
         tox_id = self._tox.group_get_chat_id(group_number)
         try:

toxygen/contacts/profile.py

@@ -1,9 +1,15 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 from contacts import basecontact
 import random
 import threading
 import common.tox_save as tox_save
 from middleware.threads import invoke_in_main_thread
 
+iUMAXINT = 4294967295
+
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
 
 class Profile(basecontact.BaseContact, tox_save.ToxSave):
     """
@@ -14,6 +20,7 @@ class Profile(basecontact.BaseContact, tox_save.ToxSave):
         :param tox: tox instance
         :param screen: ref to main screen
         """
+        assert tox
         basecontact.BaseContact.__init__(self,
                                          profile_manager,
                                          tox.self_get_name(),
@@ -60,10 +67,10 @@ class Profile(basecontact.BaseContact, tox_save.ToxSave):
 
     def set_new_nospam(self):
         """Sets new nospam part of tox id"""
-        self._tox.self_set_nospam(random.randint(0, 4294967295))  # no spam - uint32
+        self._tox.self_set_nospam(random.randint(0, iUMAXINT))  # no spam - uint32
         self._tox_id = self._tox.self_get_address()
-
+        self._sToxId = self._tox.self_get_address()
-        return self._tox_id
+        return self._sToxId
 
     # -----------------------------------------------------------------------------------------------------------------
     # Reset

toxygen/file_transfers/file_transfers_handler.py

@@ -1,11 +1,19 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 from messenger.messages import *
 from ui.contact_items import *
 import utils.util as util
 from common.tox_save import ToxSave
+from wrapper_tests.support_testing import assert_main_thread
+from copy import deepcopy
 
+# LOG=util.log
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
+log = lambda x: LOG.info(x)
 
 class FileTransfersHandler(ToxSave):
-
+    lBlockAvatars = []
     def __init__(self, tox, settings, contact_provider, file_transfers_message_service, profile):
         super().__init__(tox)
         self._settings = settings
@@ -19,6 +27,7 @@ class FileTransfersHandler(ToxSave):
         # key = (friend number, file number), value - message id
 
         profile.avatar_changed_event.add_callback(self._send_avatar_to_contacts)
+        self. lBlockAvatars = []
 
     def stop(self):
         self._settings['paused_file_transfers'] = self._paused_file_transfers if self._settings['resend_files'] else {}
@@ -37,6 +46,7 @@ class FileTransfersHandler(ToxSave):
         :param file_name: file name without path
         """
         friend = self._get_friend_by_number(friend_number)
+        if friend is None: return None
         auto = self._settings['allow_auto_accept'] and friend.tox_id in self._settings['auto_accept_from_friends']
         inline = is_inline(file_name) and self._settings['allow_inline']
         file_id = self._tox.file_get_file_id(friend_number, file_number)
@@ -85,7 +95,9 @@ class FileTransfersHandler(ToxSave):
             self._tox.file_control(friend_number, file_number, TOX_FILE_CONTROL['CANCEL'])
 
     def cancel_not_started_transfer(self, friend_number, message_id):
-        self._get_friend_by_number(friend_number).delete_one_unsent_file(message_id)
+        friend = self._get_friend_by_number(friend_number)
+        if friend is None: return None
+        friend.delete_one_unsent_file(message_id)
 
     def pause_transfer(self, friend_number, file_number, by_friend=False):
         """
@@ -115,6 +127,7 @@ class FileTransfersHandler(ToxSave):
         """
         path = self._generate_valid_path(path, from_position)
         friend = self._get_friend_by_number(friend_number)
+        if friend is None: return None
         if not inline:
             rt = ReceiveTransfer(path, self._tox, friend_number, size, file_number, from_position)
         else:
@@ -145,6 +158,7 @@ class FileTransfersHandler(ToxSave):
 
     def send_inline(self, data, file_name, friend_number, is_resend=False):
         friend = self._get_friend_by_number(friend_number)
+        if friend is None: return None
         if friend.status is None and not is_resend:
             self._file_transfers_message_service.add_unsent_file_message(friend, file_name, data)
             return
@@ -162,11 +176,12 @@ class FileTransfersHandler(ToxSave):
         :param file_id: file id of transfer
         """
         friend = self._get_friend_by_number(friend_number)
+        if friend is None: return None
         if friend.status is None and not is_resend:
             self._file_transfers_message_service.add_unsent_file_message(friend, path, None)
             return
         elif friend.status is None and is_resend:
-            print('Error in sending')
+            LOG.error('Error in sending')
             return
         st = SendTransfer(path, self._tox, friend_number, TOX_FILE_KIND['DATA'], file_id)
         file_name = os.path.basename(path)
@@ -186,23 +201,27 @@ class FileTransfersHandler(ToxSave):
 
     def transfer_finished(self, friend_number, file_number):
         transfer = self._file_transfers[(friend_number, file_number)]
+        friend = self._get_friend_by_number(friend_number)
+        if friend is None: return None
         t = type(transfer)
         if t is ReceiveAvatar:
-            self._get_friend_by_number(friend_number).load_avatar()
+            friend.load_avatar()
         elif t is ReceiveToBuffer or (t is SendFromBuffer and self._settings['allow_inline']):  # inline image
-            print('inline')
+            LOG.debug('inline')
             inline = InlineImageMessage(transfer.data)
             message_id = self._insert_inline_before[(friend_number, file_number)]
             del self._insert_inline_before[(friend_number, file_number)]
-            index = self._get_friend_by_number(friend_number).insert_inline(message_id, inline)
+            if friend is None: return None
+            index = friend.insert_inline(message_id, inline)
             self._file_transfers_message_service.add_inline_message(transfer, index)
         del self._file_transfers[(friend_number, file_number)]
 
     def send_files(self, friend_number):
+        try:
             friend = self._get_friend_by_number(friend_number)
+            if friend is None: return None
             friend.remove_invalid_unsent_files()
             files = friend.get_unsent_files()
-        try:
             for fl in files:
                 data, path = fl.data, fl.path
                 if data is not None:
@@ -211,6 +230,7 @@ class FileTransfersHandler(ToxSave):
                     self.send_file(path, friend_number, True)
             friend.clear_unsent_files()
             for key in self._paused_file_transfers.keys():
+                # RuntimeError: dictionary changed size during iteration
                 (path, ft_friend_number, is_incoming, start_position) = self._paused_file_transfers[key]
                 if not os.path.exists(path):
                     del self._paused_file_transfers[key]
@@ -218,12 +238,16 @@ class FileTransfersHandler(ToxSave):
                     self.send_file(path, friend_number, True, key)
                     del self._paused_file_transfers[key]
         except Exception as ex:
-            print('Exception in file sending: ' + str(ex))
+            LOG.error('Exception in file sending: ' + str(ex))
 
     def friend_exit(self, friend_number):
-        for friend_num, file_num in self._file_transfers.keys():
+        # RuntimeError: dictionary changed size during iteration
+        lMayChangeDynamically = self._file_transfers.copy()
+        for friend_num, file_num in lMayChangeDynamically:
             if friend_num != friend_number:
                 continue
+            if (friend_num, file_num) not in self._file_transfers:
+                continue
             ft = self._file_transfers[(friend_num, file_num)]
             if type(ft) is SendTransfer:
                 self._paused_file_transfers[ft.file_id] = [ft.path, friend_num, False, -1]
@@ -240,8 +264,16 @@ class FileTransfersHandler(ToxSave):
         :param friend_number: number of friend who should get new avatar
         :param avatar_path: path to avatar or None if reset
         """
+        if (avatar_path, friend_number,) in self.lBlockAvatars:
+            return
+
+        try:
             sa = SendAvatar(avatar_path, self._tox, friend_number)
             self._file_transfers[(friend_number, sa.file_number)] = sa
+        except Exception as e:
+            # ArgumentError('This client is currently not connected to the friend.')
+            LOG.error(f"send_avatar {e}")
+            self.lBlockAvatars.append( (avatar_path, friend_number,) )
 
     def incoming_avatar(self, friend_number, file_number, size):
         """
@@ -251,6 +283,7 @@ class FileTransfersHandler(ToxSave):
         :param size: size of avatar or 0 (default avatar)
         """
         friend = self._get_friend_by_number(friend_number)
+        if friend is None: return None
         ra = ReceiveAvatar(friend.get_contact_avatar_path(), self._tox, friend_number, size, file_number)
         if ra.state != FILE_TRANSFER_STATE['CANCELLED']:
             self._file_transfers[(friend_number, file_number)] = ra
@@ -259,6 +292,7 @@ class FileTransfersHandler(ToxSave):
             friend.reset_avatar(self._settings['identicons'])
 
     def _send_avatar_to_contacts(self, _):
+        # from a callback
         friends = self._get_all_friends()
         for friend in filter(self._is_friend_online, friends):
             self.send_avatar(friend.number)
@@ -269,6 +303,7 @@ class FileTransfersHandler(ToxSave):
 
     def _is_friend_online(self, friend_number):
         friend = self._get_friend_by_number(friend_number)
+        if friend is None: return None
 
         return friend.status is not None
 

toxygen/file_transfers/file_transfers_messages_service.py

@@ -2,6 +2,15 @@ from messenger.messenger import *
 import utils.util as util
 from file_transfers.file_transfers import *
 
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
+
+def LOG_ERROR(l): print('ERROR_: '+l)
+def LOG_WARN(l): print('WARN_: '+l)
+def LOG_INFO(l): print('INFO_: '+l)
+def LOG_DEBUG(l): print('DEBUG_: '+l)
+def LOG_TRACE(l): pass # print('TRACE+ '+l)
 
 class FileTransfersMessagesService:
 
@@ -12,6 +21,7 @@ class FileTransfersMessagesService:
         self._messages = main_screen.messages
 
     def add_incoming_transfer_message(self, friend, accepted, size, file_name, file_number):
+        assert friend
         author = MessageAuthor(friend.name, MESSAGE_AUTHOR['FRIEND'])
         status = FILE_TRANSFER_STATE['RUNNING'] if accepted else FILE_TRANSFER_STATE['INCOMING_NOT_STARTED']
         tm = TransferMessage(author, util.get_unix_time(), status, size, file_name, friend.number, file_number)
@@ -27,6 +37,7 @@ class FileTransfersMessagesService:
         return tm
 
     def add_outgoing_transfer_message(self, friend, size, file_name, file_number):
+        assert friend
         author = MessageAuthor(self._profile.name, MESSAGE_AUTHOR['ME'])
         status = FILE_TRANSFER_STATE['OUTGOING_NOT_STARTED']
         tm = TransferMessage(author, util.get_unix_time(), status, size, file_name, friend.number, file_number)
@@ -40,13 +51,19 @@ class FileTransfersMessagesService:
         return tm
 
     def add_inline_message(self, transfer, index):
+        """callback"""
         if not self._is_friend_active(transfer.friend_number):
             return
+        if transfer is None or not hasattr(transfer, 'data') or \
+           not transfer.data:
+            LOG_ERROR(f"add_inline_message empty data")
+            return
         count = self._messages.count()
         if count + index + 1 >= 0:
             self._create_inline_item(transfer.data, count + index + 1)
 
     def add_unsent_file_message(self, friend, file_path, data):
+        assert friend
         author = MessageAuthor(self._profile.name, MESSAGE_AUTHOR['ME'])
         size = os.path.getsize(file_path) if data is None else len(data)
         tm = UnsentFileMessage(file_path, data, util.get_unix_time(), author, size, friend.number)

toxygen/groups/group_peer.py

@@ -13,7 +13,8 @@ class GroupChatPeer:
         self._public_key = public_key
         self._is_current_user = is_current_user
         self._is_muted = is_muted
-
+        # unused?
+        self._kind = 'grouppeer'
     # -----------------------------------------------------------------------------------------------------------------
     # Readonly properties
     # -----------------------------------------------------------------------------------------------------------------

toxygen/groups/groups_service.py

@@ -1,9 +1,16 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+
 import common.tox_save as tox_save
 import utils.ui as util_ui
 from groups.peers_list import PeersListGenerator
 from groups.group_invite import GroupInvite
 import wrapper.toxcore_enums_and_consts as constants
+from wrapper.toxcore_enums_and_consts import *
+from wrapper.tox import UINT32_MAX
 
+global LOG
+import logging
+LOG = logging.getLogger('app.'+'gs')
 
 class GroupsService(tox_save.ToxSave):
 
@@ -16,6 +23,8 @@ class GroupsService(tox_save.ToxSave):
         self._widgets_factory_provider = widgets_factory_provider
         self._group_invites = []
         self._screen = None
+        # maybe just use self
+        self._tox = tox
 
     def set_tox(self, tox):
         super().set_tox(tox)
@@ -27,7 +36,11 @@ class GroupsService(tox_save.ToxSave):
     # -----------------------------------------------------------------------------------------------------------------
 
     def create_new_gc(self, name, privacy_state, nick, status):
+        try:
             group_number = self._tox.group_new(privacy_state, name, nick, status)
+        except Exception as e:
+            LOG.error(f"create_new_gc {e}")
+            return
         if group_number == -1:
             return
 
@@ -37,14 +50,36 @@ class GroupsService(tox_save.ToxSave):
         self._contacts_manager.update_filtration()
 
     def join_gc_by_id(self, chat_id, password, nick, status):
+        try:
             group_number = self._tox.group_join(chat_id, password, nick, status)
+            assert type(group_number) == int, group_number
+            assert group_number < UINT32_MAX, group_number
+        except Exception as e:
+            # gui
+            title = f"join_gc_by_id {chat_id}"
+            util_ui.message_box(title +'\n' +str(e), title)
+            LOG.error(f"_join_gc_via_id {e}")
+            return
+        LOG.debug(f"_join_gc_via_id {group_number}")
         self._add_new_group_by_number(group_number)
+        group = self._get_group_by_number(group_number)
+        try:
+            assert group and hasattr(group, 'status')
+        except Exception as e:
+            # gui
+            title = f"join_gc_by_id {chat_id}"
+            util_ui.message_box(title +'\n' +str(e), title)
+            LOG.error(f"_join_gc_via_id {e}")
+            return
+        group.status = constants.TOX_USER_STATUS['NONE']
+        self._contacts_manager.update_filtration()
 
     # -----------------------------------------------------------------------------------------------------------------
     # Groups reconnect and leaving
     # -----------------------------------------------------------------------------------------------------------------
 
     def leave_group(self, group_number):
+        if type(group_number) == int:
             self._tox.group_leave(group_number)
             self._contacts_manager.delete_group(group_number)
 
@@ -65,10 +100,22 @@ class GroupsService(tox_save.ToxSave):
     # -----------------------------------------------------------------------------------------------------------------
 
     def invite_friend(self, friend_number, group_number):
+        if self._tox.friend_get_connection_status(friend_number) == TOX_CONNECTION['NONE']:
+            title = f"Error in group_invite_friend {friend_number}"
+            e = f"Friend not connected friend_number={friend_number}"
+            util_ui.message_box(title +'\n' +str(e), title)
+            return
+
+        try:
             self._tox.group_invite_friend(group_number, friend_number)
+        except Exception as e:
+            title = f"Error in group_invite_friend {group_number} {friend_number}"
+            util_ui.message_box(title +'\n' +str(e), title)
 
     def process_group_invite(self, friend_number, group_name, invite_data):
         friend = self._get_friend_by_number(friend_number)
+        # binary  {invite_data}
+        LOG.debug(f"process_group_invite {friend_number} {group_name}")
         invite = GroupInvite(friend.tox_id, group_name, invite_data)
         self._group_invites.append(invite)
         self._update_invites_button_state()
@@ -76,6 +123,7 @@ class GroupsService(tox_save.ToxSave):
     def accept_group_invite(self, invite, name, status, password):
         pk = invite.friend_public_key
         friend = self._get_friend_by_public_key(pk)
+        LOG.debug(f"accept_group_invite {name}")
         self._join_gc_via_invite(invite.invite_data, friend.number, name, status, password)
         self._delete_group_invite(invite)
         self._update_invites_button_state()
@@ -188,6 +236,7 @@ class GroupsService(tox_save.ToxSave):
     # -----------------------------------------------------------------------------------------------------------------
 
     def show_bans_list(self, group):
+        return
         widgets_factory = self._get_widgets_factory()
         self._screen = widgets_factory.create_groups_bans_screen(group)
         self._screen.show()
@@ -206,6 +255,7 @@ class GroupsService(tox_save.ToxSave):
     # -----------------------------------------------------------------------------------------------------------------
 
     def _add_new_group_by_number(self, group_number):
+        LOG.debug(f"_add_new_group_by_number {group_number}")
         self._contacts_manager.add_group(group_number)
 
     def _get_group_by_number(self, group_number):
@@ -232,8 +282,21 @@ class GroupsService(tox_save.ToxSave):
             self._group_invites.remove(invite)
 
     def _join_gc_via_invite(self, invite_data, friend_number, nick, status, password):
+        LOG.debug(f"_join_gc_via_invite friend_number={friend_number} nick={nick} datalen={len(invite_data)}")
+        if nick is None:
+            nick = ''
+        if invite_data is None:
+            invite_data = b''
+        try:
             group_number = self._tox.group_invite_accept(invite_data, friend_number, nick, status, password)
+        except Exception as e:
+            LOG.error(f"_join_gc_via_invite ERROR {e}")
+            return
+        try:
             self._add_new_group_by_number(group_number)
+        except Exception as e:
+            LOG.error(f"_join_gc_via_invite group_number={group_number} {e}")
+            return
 
     def _update_invites_button_state(self):
         self._main_screen.update_gc_invites_button_state()

toxygen/history/database.py

@@ -1,19 +1,20 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 from sqlite3 import connect
 import os.path
 import utils.util as util
 
+global LOG
+import logging
+LOG = logging.getLogger('app.db')
 
 TIMEOUT = 11
-
 SAVE_MESSAGES = 500
-
 MESSAGE_AUTHOR = {
     'ME': 0,
     'FRIEND': 1,
     'NOT_SENT': 2,
     'GC_PEER': 3
 }
-
 CONTACT_TYPE = {
     'FRIEND': 0,
     'GC_PEER': 1,
@@ -24,19 +25,31 @@ CONTACT_TYPE = {
 class Database:
 
     def __init__(self, path, toxes):
-        self._path, self._toxes = path, toxes
+        self._path = path
+        self._toxes = toxes
         self._name = os.path.basename(path)
-        if os.path.exists(path):
+
+    def open(self):
+        path = self._path
+        toxes = self._toxes
+        if not os.path.exists(path):
+            LOG.warn('Db not found: ' +path)
+            return
         try:
             with open(path, 'rb') as fin:
                 data = fin.read()
+        except Exception as ex:
+            LOG.error('Db reading error: ' +path +' ' +str(ex))
+            raise
+        try:
             if toxes.is_data_encrypted(data):
                 data = toxes.pass_decrypt(data)
                 with open(path, 'wb') as fout:
                     fout.write(data)
         except Exception as ex:
-                util.log('Db reading error: ' + str(ex))
+            LOG.error('Db writing error: ' +path +' ' + str(ex))
             os.remove(path)
+        LOG.info('Db opened: ' +path)
 
     # -----------------------------------------------------------------------------------------------------------------
     # Public methods
@@ -58,6 +71,7 @@ class Database:
             data = self._toxes.pass_encrypt(data)
         with open(new_path, 'wb') as fout:
             fout.write(data)
+        LOG.info('Db exported: ' +new_path)
 
     def add_friend_to_db(self, tox_id):
         db = self._connect()
@@ -72,11 +86,14 @@ class Database:
                            '    message_type INTEGER'
                            ')')
             db.commit()
-        except:
+            return True
-            print('Database is locked!')
+        except Exception as e:
+            LOG.error("dd_friend_to_db " +self._name +' Database exception! ' +str(e))
             db.rollback()
+            return False
         finally:
             db.close()
+            LOG.debug(f"add_friend_to_db {tox_id}")
 
     def delete_friend_from_db(self, tox_id):
         db = self._connect()
@@ -84,11 +101,14 @@ class Database:
             cursor = db.cursor()
             cursor.execute('DROP TABLE id' + tox_id + ';')
             db.commit()
-        except:
+            return True
-            print('Database is locked!')
+        except Exception as e:
+            LOG.error("delete_friend_from_db " +self._name +' Database exception! ' +str(e))
             db.rollback()
+            return False
         finally:
             db.close()
+            LOG.debug(f"delete_friend_from_db {tox_id}")
 
     def save_messages_to_db(self, tox_id, messages_iter):
         db = self._connect()
@@ -96,13 +116,16 @@ class Database:
             cursor = db.cursor()
             cursor.executemany('INSERT INTO id' + tox_id +
                                '(message, author_name, author_type, unix_time, message_type) ' +
-                               'VALUES (?, ?, ?, ?, ?, ?);', messages_iter)
+                               'VALUES (?, ?, ?, ?, ?);', messages_iter)
             db.commit()
-        except:
+            return True
-            print('Database is locked!')
+        except Exception as e:
+            LOG.error("" +self._name +' Database exception! ' +str(e))
             db.rollback()
+            return False
         finally:
             db.close()
+            LOG.debug(f"save_messages_to_db {tox_id}")
 
     def update_messages(self, tox_id, message_id):
         db = self._connect()
@@ -111,11 +134,14 @@ class Database:
             cursor.execute('UPDATE id' + tox_id + ' SET author = 0 '
                            'WHERE id = ' + str(message_id) + ' AND author = 2;')
             db.commit()
-        except:
+            return True
-            print('Database is locked!')
+        except Exception as e:
+            LOG.error("" +self._name +' Database exception! ' +str(e))
             db.rollback()
+            return False
         finally:
             db.close()
+            LOG.debug(f"update_messages {tox_id}")
 
     def delete_message(self, tox_id, unique_id):
         db = self._connect()
@@ -123,11 +149,14 @@ class Database:
             cursor = db.cursor()
             cursor.execute('DELETE FROM id' + tox_id + ' WHERE id = ' + str(unique_id) + ';')
             db.commit()
-        except:
+            return True
-            print('Database is locked!')
+        except Exception as e:
+            LOG.error("" +self._name +' Database exception! ' +str(e))
             db.rollback()
+            return False
         finally:
             db.close()
+            LOG.debug(f"delete_message {tox_id}")
 
     def delete_messages(self, tox_id):
         db = self._connect()
@@ -135,11 +164,14 @@ class Database:
             cursor = db.cursor()
             cursor.execute('DELETE FROM id' + tox_id + ';')
             db.commit()
-        except:
+            return True
-            print('Database is locked!')
+        except Exception as e:
+            LOG.error("" +self._name +' Database exception! ' +str(e))
             db.rollback()
+            return False
         finally:
             db.close()
+            LOG.debug(f"delete_messages {tox_id}")
 
     def messages_getter(self, tox_id):
         self.add_friend_to_db(tox_id)

toxygen/history/history.py

@@ -1,5 +1,9 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 from history.history_logs_generators import *
 
+global LOG
+import logging
+LOG = logging.getLogger('app.db')
 
 class History:
 
@@ -26,7 +30,8 @@ class History:
         """
         Save history to db
         """
-        if self._settings['save_db']:
+        # me a mistake? was _db not _history
+        if self._settings['save_history']:
             for friend in self._contact_provider.get_all_friends():
                 self._db.add_friend_to_db(friend.tox_id)
                 if not self._settings['save_unsent_only']:
@@ -57,8 +62,10 @@ class History:
             file_name += '.' + extension
 
         history = self.generate_history(contact, as_text)
+        assert history
         with open(file_name, 'wt') as fl:
             fl.write(history)
+        LOG.info(f"wrote history to {file_name}")
         
     def delete_message(self, message):
         contact = self._contacts_manager.get_curr_contact()

toxygen/main.py

@@ -1,51 +1,404 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+import sys
+import os
 import app
-from user_data.settings import *
-import utils.util as util
 import argparse
+import logging
+import signal
 
+import faulthandler
+faulthandler.enable()
+
+import warnings
+warnings.filterwarnings('ignore')
+
+import wrapper_tests.support_testing as ts
+try:
+    from trepan.interfaces import server as Mserver
+    from trepan.api import debug
+except:
+    print('trepan3 TCP server NOT enabled.')
+else:
+    import signal
+    try:
+        signal.signal(signal.SIGUSR1, ts.trepan_handler)
+        print('trepan3 TCP server enabled on port 6666.')
+    except: pass
+
+from user_data.settings import *
+from user_data.settings import Settings
+from user_data import settings
+import utils.util as util
+with ts.ignoreStderr():
+    import pyaudio
 
 __maintainer__ = 'Ingvar'
-__version__ = '0.5.0'
+__version__ = '0.5.0+'
 
+import time
+sleep = time.sleep
+
+def reset():
+    Settings.reset_auto_profile()
 
 def clean():
     """Removes libs folder"""
     directory = util.get_libs_directory()
     util.remove(directory)
 
-
-def reset():
-    Settings.reset_auto_profile()
-
-
 def print_toxygen_version():
-    print('Toxygen v' + __version__)
+    print('Toxygen ' + __version__)
 
+def setup_default_audio():
+    # need:
+    audio = ts.get_audio()
+    # unfinished
+    global oPYA
+    oPYA = pyaudio.PyAudio()
+    audio['output_devices'] = dict()
+    i = oPYA.get_device_count()
+    while i > 0:
+        i -= 1
+        if oPYA.get_device_info_by_index(i)['maxOutputChannels'] == 0:
+            continue
+        audio['output_devices'][i] = oPYA.get_device_info_by_index(i)['name']
+    i = oPYA.get_device_count()
+    audio['input_devices'] = dict()
+    while i > 0:
+        i -= 1
+        if oPYA.get_device_info_by_index(i)['maxInputChannels'] == 0:
+            continue
+        audio['input_devices'][i] = oPYA.get_device_info_by_index(i)['name']
+    return audio
 
-def main():
+def setup_video(oArgs):
+    video = setup_default_video()
+    if oArgs.video_input == '-1':
+        video['device'] = video['output_devices'][1]
+    else:
+        video['device'] = oArgs.video_input
+    return video
+
+def setup_audio(oArgs):
+    global oPYA
+    audio = setup_default_audio()
+    for k,v in audio['input_devices'].items():
+        if v == 'default' and 'input' not in audio:
+            audio['input'] = k
+        if v == getattr(oArgs, 'audio_input'):
+            audio['input'] = k
+            LOG.debug(f"Setting audio['input'] {k} = {v} {k}")
+            break
+    for k,v in audio['output_devices'].items():
+        if v == 'default' and 'output' not in audio:
+            audio['output'] = k
+        if v == getattr(oArgs, 'audio_output'):
+            audio['output'] = k
+            LOG.debug(f"Setting audio['output'] {k} = {v} " +str(k))
+            break
+
+    if hasattr(oArgs, 'mode') and getattr(oArgs, 'mode') > 1:
+        audio['enabled'] = True
+        audio['audio_enabled'] = True
+        audio['video_enabled'] = True
+    elif hasattr(oArgs, 'mode') and getattr(oArgs, 'mode') > 0:
+        audio['enabled'] = True
+        audio['audio_enabled'] = False
+        audio['video_enabled'] = True
+    else:
+        audio['enabled'] = False
+        audio['audio_enabled'] = False
+        audio['video_enabled'] = False
+
+    return audio
+
+    i = getattr(oArgs, 'audio_output')
+    if i >= 0:
+        try:
+            elt = oPYA.get_device_info_by_index(i)
+            if i >= 0 and ( 'maxOutputChannels' not in elt or \
+                            elt['maxOutputChannels'] == 0):
+                LOG.warn(f"Audio output device has no output channels:  {i}")
+                oArgs.audio_output = -1
+        except OSError as e:
+            LOG.warn("Audio output device error looking for maxOutputChannels: " \
+                      +str(i) +' ' +str(e))
+            oArgs.audio_output = -1
+
+    if getattr(oArgs, 'audio_output') < 0:
+        LOG.info("Choose an output device:")
+        i = oPYA.get_device_count()
+        while i > 0:
+            i -= 1
+            if oPYA.get_device_info_by_index(i)['maxOutputChannels'] == 0:
+                continue
+            LOG.info(str(i) \
+                  +' ' +oPYA.get_device_info_by_index(i)['name'] \
+                  +' ' +str(oPYA.get_device_info_by_index(i)['defaultSampleRate'])
+                  )
+        return 0
+
+    i = getattr(oArgs, 'audio_input')
+    if i >= 0:
+        try:
+            elt = oPYA.get_device_info_by_index(i)
+            if i >= 0 and ( 'maxInputChannels' not in elt or \
+                            elt['maxInputChannels'] == 0):
+                LOG.warn(f"Audio input device has no input channels:  {i}")
+                setattr(oArgs, 'audio_input', -1)
+        except OSError as e:
+            LOG.warn("Audio input device error looking for maxInputChannels: " \
+                      +str(i) +' ' +str(e))
+            setattr(oArgs, 'audio_input', -1)
+    if getattr(oArgs, 'audio_input') < 0:
+        LOG.info("Choose an input device:")
+        i = oPYA.get_device_count()
+        while i > 0:
+            i -= 1
+            if oPYA.get_device_info_by_index(i)['maxInputChannels'] == 0:
+                continue
+            LOG.info(str(i) \
+                     +' ' +oPYA.get_device_info_by_index(i)['name']
+                     +' ' +str(oPYA.get_device_info_by_index(i)['defaultSampleRate'])
+                  )
+        return 0
+
+def setup_default_video():
+    default_video = ["-1"]
+    default_video.extend(ts.get_video_indexes())
+    LOG.info(f"Video input choices: {default_video!r}")
+    video = {'device': -1, 'width': 320, 'height': 240, 'x': 0, 'y': 0}
+    video['output_devices'] = default_video
+    return video
+
+def main_parser():
+    import cv2
+    if not os.path.exists('/proc/sys/net/ipv6'):
+        bIpV6 = 'False'
+    else:
+        bIpV6 = 'True'
+    lIpV6Choices=[bIpV6, 'False']
+
+    audio = setup_default_audio()
+    default_video = setup_default_video()
+
+    logfile = os.path.join(os.environ.get('TMPDIR', '/tmp'), 'toxygen.log')
     parser = argparse.ArgumentParser()
     parser.add_argument('--version', action='store_true', help='Prints Toxygen version')
     parser.add_argument('--clean', action='store_true', help='Delete toxcore libs from libs folder')
     parser.add_argument('--reset', action='store_true', help='Reset default profile')
-    parser.add_argument('--uri', help='Add specified Tox ID to friends')
+    parser.add_argument('--uri', type=str, default='',
-    parser.add_argument('profile', nargs='?', default=None, help='Path to Tox profile')
+                        help='Add specified Tox ID to friends')
-    args = parser.parse_args()
+    parser.add_argument('--logfile', default=logfile,
+                        help='Filename for logging')
+    parser.add_argument('--loglevel', type=int, default=logging.INFO,
+                        help='Threshold for logging (lower is more) default: 20')
+    parser.add_argument('--proxy_host', '--proxy-host', type=str,
+                        # oddball - we want to use '' as a setting
+                        default='0.0.0.0',
+                        help='proxy host')
+    parser.add_argument('--proxy_port', '--proxy-port', default=0, type=int,
+                        help='proxy port')
+    parser.add_argument('--proxy_type', '--proxy-type', default=0, type=int,
+                        choices=[0,1,2],
+                        help='proxy type 1=https, 2=socks')
+    parser.add_argument('--tcp_port', '--tcp-port', default=0, type=int,
+                        help='tcp port')
+    parser.add_argument('--auto_accept_path', '--auto-accept-path', type=str,
+                        default=os.path.join(os.environ['HOME'], 'Downloads'),
+                        help="auto_accept_path")
+    parser.add_argument('--mode', type=int, default=2,
+                        help='Mode: 0=chat 1=chat+audio 2=chat+audio+video default: 0')
+    parser.add_argument('--font', type=str, default="Courier",
+                        help='Message font')
+    parser.add_argument('--message_font_size', type=int, default=15,
+                        help='Font size in pixels')
+    parser.add_argument('--local_discovery_enabled',type=str,
+                        default='False', choices=['True','False'],
+                        help='Look on the local lan')
+    parser.add_argument('--udp_enabled',type=str,
+                        default='True', choices=['True','False'],
+                        help='En/Disable udp')
+    parser.add_argument('--trace_enabled',type=str,
+                        default='False', choices=['True','False'],
+                        help='Debugging from toxcore logger_trace')
+    parser.add_argument('--ipv6_enabled',type=str,
+                        default=bIpV6, choices=lIpV6Choices,
+                        help='En/Disable ipv6')
+    parser.add_argument('--compact_mode',type=str,
+                        default='True', choices=['True','False'],
+                        help='Compact mode')
+    parser.add_argument('--allow_inline',type=str,
+                        default='False', choices=['True','False'],
+                        help='Dis/Enable allow_inline')
+    parser.add_argument('--notifications',type=str,
+                        default='True', choices=['True','False'],
+                        help='Dis/Enable notifications')
+    parser.add_argument('--sound_notifications',type=str,
+                        default='True', choices=['True','False'],
+                        help='Enable sound notifications')
+    parser.add_argument('--calls_sound',type=str,
+                        default='True', choices=['True','False'],
+                        help='Enable calls_sound')
+    parser.add_argument('--core_logging',type=str,
+                        default='False', choices=['True','False'],
+                        help='Dis/Enable Toxcore notifications')
+    parser.add_argument('--hole_punching_enabled',type=str,
+                        default='False', choices=['True','False'],
+                        help='En/Enable hole punching')
+    parser.add_argument('--dht_announcements_enabled',type=str,
+                        default='True', choices=['True','False'],
+                        help='En/Disable DHT announcements')
+    parser.add_argument('--save_history',type=str,
+                        default='True', choices=['True','False'],
+                        help='En/Disable save history')
+    parser.add_argument('--update', type=int, default=0,
+                        choices=[0,0],
+                        help='Update program (broken)')
+    parser.add_argument('--download_nodes_list',type=str,
+                        default='False', choices=['True','False'],
+                        help='Download nodes list')
+    parser.add_argument('--nodes_json', type=str,
+                        default='')
+    parser.add_argument('--download_nodes_url', type=str,
+                        default='https://nodes.tox.chat/json')
+    parser.add_argument('--network', type=str,
+                        choices=['old', 'main', 'new', 'local', 'newlocal'],
+                        default='old')
+    parser.add_argument('--video_input', type=str,
+                        default=-1,
+                        choices=default_video['output_devices'],
+                        help="Video input device number - /dev/video?")
+    parser.add_argument('--audio_input', type=str,
+                        default=oPYA.get_default_input_device_info()['name'],
+                        choices=audio['input_devices'].values(),
+                        help="Audio input device name - aplay -L for help")
+    parser.add_argument('--audio_output', type=str,
+                        default=oPYA.get_default_output_device_info()['index'],
+                        choices=audio['output_devices'].values(),
+                        help="Audio output device number - -1 for help")
+    parser.add_argument('--theme', type=str, default='default',
+                        choices=['dark', 'default'],
+                        help='Theme - style of UI')
+    parser.add_argument('--sleep', type=str, default='time',
+                        # could expand this to tk, gtk, gevent...
+                        choices=['qt','gevent','time'],
+                        help='Sleep method - one of qt, gevent , time')
+    supported_languages = settings.supported_languages()
+    parser.add_argument('--language', type=str, default='English',
+                        choices=supported_languages,
+                        help='Languages')
+    parser.add_argument('profile', type=str, nargs='?', default=None,
+                        help='Path to Tox profile')
+    return parser
 
-    if args.version:
+# clean out the unchanged settings so these can override the profile
+lKEEP_SETTINGS = ['uri',
+                  'profile',
+                  'loglevel',
+                  'logfile',
+                  'mode',
+
+                  # dunno
+                  'audio_input',
+                  'audio_output',
+                  'audio',
+                  'video',
+
+                  'ipv6_enabled',
+                  'udp_enabled',
+                  'local_discovery_enabled',
+                  'theme',
+                  'network',
+                  'message_font_size',
+                  'font',
+                  'save_history',
+                  'language',
+                  'update',
+                  'proxy_host',
+                  'proxy_type',
+                  'proxy_port',
+                  'core_logging',
+                  'audio',
+                  'video'
+                  ] # , 'nodes_json'
+
+class A(): pass
+
+def main(lArgs):
+    global oPYA
+    from argparse import Namespace
+    parser = main_parser()
+    default_ns = parser.parse_args([])
+    oArgs = parser.parse_args(lArgs)
+
+    if oArgs.version:
         print_toxygen_version()
-        return
+        return 0
 
-    if args.clean:
+    if oArgs.clean:
         clean()
-        return
+        return 0
 
-    if args.reset:
+    if oArgs.reset:
         reset()
-        return
+        return 0
 
-    toxygen = app.App(__version__, args.profile, args.uri)
+    # if getattr(oArgs, 'network') in ['newlocal', 'localnew']: oArgs.network = 'new'
-    toxygen.main()
 
+    # clean out the unchanged settings so these can override the profile
+    for key in default_ns.__dict__.keys():
+        if key in lKEEP_SETTINGS: continue
+        if not hasattr(oArgs, key): continue
+        if getattr(default_ns, key) == getattr(oArgs, key):
+            delattr(oArgs, key)
+
+    for key in ts.lBOOLEANS:
+        if not hasattr(oArgs, key): continue
+        val = getattr(oArgs, key)
+        if type(val) == bool: continue
+        if val in ['False', 'false', '0']:
+            setattr(oArgs, key, False)
+        else:
+            setattr(oArgs, key, True)
+
+    aArgs = A()
+    for key in oArgs.__dict__.keys():
+        setattr(aArgs, key, getattr(oArgs, key))
+    #setattr(aArgs, 'video', setup_video(oArgs))
+    aArgs.video = setup_video(oArgs)
+    assert 'video' in aArgs.__dict__
+
+    #setattr(aArgs, 'audio', setup_audio(oArgs))
+    aArgs.audio = setup_audio(oArgs)
+    assert 'audio' in aArgs.__dict__
+    oArgs = aArgs
+
+    toxygen = app.App(__version__, oArgs)
+    # for pyqtconsole
+    __builtins__.app = toxygen
+    i = toxygen.iMain()
+    return i
 
 if __name__ == '__main__':
-    main()
+    iRet = 0
+    try:
+        iRet = main(sys.argv[1:])
+    except KeyboardInterrupt:
+        iRet = 0
+    except SystemExit as e:
+        iRet = e
+    except Exception as e:
+        import traceback
+        sys.stderr.write(f"Exception from main  {e}" \
+                         +'\n' + traceback.format_exc() +'\n' )
+        iRet = 1
+
+    # Exception ignored in: <module 'threading' from '/usr/lib/python3.9/threading.py'>
+    # File "/usr/lib/python3.9/threading.py", line 1428, in _shutdown
+    # lock.acquire()
+    # gevent.exceptions.LoopExit as e:
+    # This operation would block forever
+    sys.stderr.write('Calling sys.exit' +'\n')
+    with ts.ignoreStdout():
+        sys.exit(iRet)

toxygen/messenger/messages.py

@@ -38,8 +38,8 @@ class Message:
 
     MESSAGE_ID = 0
 
-    def __init__(self, message_type, author, time):
+    def __init__(self, message_type, author, iTime):
-        self._time = time
+        self._time = iTime
         self._type = message_type
         self._author = author
         self._widget = None
@@ -66,6 +66,7 @@ class Message:
     message_id = property(get_message_id)
 
     def get_widget(self, *args):
+        # FixMe
         self._widget = self._create_widget(*args)
 
         return self._widget
@@ -81,6 +82,7 @@ class Message:
             self._widget.mark_as_sent()
 
     def _create_widget(self, *args):
+        # overridden
         pass
 
     @staticmethod
@@ -95,8 +97,8 @@ class TextMessage(Message):
     Plain text or action message
     """
 
-    def __init__(self, message, owner, time, message_type, message_id=0):
+    def __init__(self, message, owner, iTime, message_type, message_id=0):
-        super().__init__(message_type, owner, time)
+        super().__init__(message_type, owner, iTime)
         self._message = message
         self._id = message_id
 
@@ -119,8 +121,8 @@ class TextMessage(Message):
 
 class OutgoingTextMessage(TextMessage):
 
-    def __init__(self, message, owner, time, message_type, tox_message_id=0):
+    def __init__(self, message, owner, iTime, message_type, tox_message_id=0):
-        super().__init__(message, owner, time, message_type)
+        super().__init__(message, owner, iTime, message_type)
         self._tox_message_id = tox_message_id
 
     def get_tox_message_id(self):
@@ -134,8 +136,8 @@ class OutgoingTextMessage(TextMessage):
 
 class GroupChatMessage(TextMessage):
 
-    def __init__(self, id, message, owner, time, message_type, name):
+    def __init__(self, id, message, owner, iTime, message_type, name):
-        super().__init__(id, message, owner, time, message_type)
+        super().__init__(id, message, owner, iTime, message_type)
         self._user_name = name
 
 
@@ -144,8 +146,8 @@ class TransferMessage(Message):
     Message with info about file transfer
     """
 
-    def __init__(self, author, time, state, size, file_name, friend_number, file_number):
+    def __init__(self, author, iTime, state, size, file_name, friend_number, file_number):
-        super().__init__(MESSAGE_TYPE['FILE_TRANSFER'], author, time)
+        super().__init__(MESSAGE_TYPE['FILE_TRANSFER'], author, iTime)
         self._state = state
         self._size = size
         self._file_name = file_name
@@ -185,10 +187,10 @@ class TransferMessage(Message):
 
     file_name = property(get_file_name)
 
-    def transfer_updated(self, state, percentage, time):
+    def transfer_updated(self, state, percentage, iTime):
         self._state = state
         if self._widget is not None:
-            self._widget.update_transfer_state(state, percentage, time)
+            self._widget.update_transfer_state(state, percentage, iTime)
 
     def _create_widget(self, *args):
         return FileTransferItem(self, *args)
@@ -196,9 +198,9 @@ class TransferMessage(Message):
 
 class UnsentFileMessage(TransferMessage):
 
-    def __init__(self, path, data, time, author, size, friend_number):
+    def __init__(self, path, data, iTime, author, size, friend_number):
         file_name = os.path.basename(path)
-        super().__init__(author, time, FILE_TRANSFER_STATE['UNSENT'], size, file_name, friend_number, -1)
+        super().__init__(author, iTime, FILE_TRANSFER_STATE['UNSENT'], size, file_name, friend_number, -1)
         self._data, self._path = data, path
 
     def get_data(self):
@@ -235,5 +237,5 @@ class InlineImageMessage(Message):
 
 class InfoMessage(TextMessage):
 
-    def __init__(self, message, time):
+    def __init__(self, message, iTime):
-        super().__init__(message, None, time, MESSAGE_TYPE['INFO_MESSAGE'])
+        super().__init__(message, None, iTime, MESSAGE_TYPE['INFO_MESSAGE'])

toxygen/messenger/messenger.py

@@ -1,6 +1,15 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 import common.tox_save as tox_save
-from messenger.messages import *
+import utils.ui as util_ui
 
+from messenger.messages import *
+from wrapper_tests.support_testing import assert_main_thread
+from wrapper.toxcore_enums_and_consts import TOX_MAX_MESSAGE_LENGTH
+
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
+log = lambda x: LOG.info(x)
 
 class Messenger(tox_save.ToxSave):
 
@@ -19,6 +28,9 @@ class Messenger(tox_save.ToxSave):
         calls_manager.call_started_event.add_callback(self._on_call_started)
         calls_manager.call_finished_event.add_callback(self._on_call_finished)
 
+    def __repr__(self):
+        return "<Messenger>"
+    
     def get_last_message(self):
         contact = self._contacts_manager.get_curr_contact()
         if contact is None:
@@ -51,33 +63,53 @@ class Messenger(tox_save.ToxSave):
             self._screen.messageEdit.clear()
             return
 
+        message_type = TOX_MESSAGE_TYPE['NORMAL']
+        if False: # undocumented
             action_message_prefix = '/me '
             if text.startswith(action_message_prefix):
                 message_type = TOX_MESSAGE_TYPE['ACTION']
                 text = text[len(action_message_prefix):]
-        else:
-            message_type = TOX_MESSAGE_TYPE['NORMAL']
 
+        if len(text) > TOX_MAX_MESSAGE_LENGTH:
+            text = text[:TOX_MAX_MESSAGE_LENGTH] # 1372
+        try:
             if self._contacts_manager.is_active_a_friend():
                 self.send_message_to_friend(text, message_type)
             elif self._contacts_manager.is_active_a_group():
                 self.send_message_to_group(text, message_type)
             elif self._contacts_manager.is_active_a_group_chat_peer():
                 self.send_message_to_group_peer(text, message_type)
+            else:
+                LOG.warn(f'Unknown friend type for Messenger send_message')
+        except Exception as e:
+            LOG.error(f'Messenger send_message {e}')
+            import traceback
+            LOG.warn(traceback.format_exc())
+            title = 'Messenger send_message Error'
+            text = 'Error: ' + str(e)
+            assert_main_thread()
+            util_ui.message_box(text, title)
             
     def send_message_to_friend(self, text, message_type, friend_number=None):
         """
         Send message
         :param text: message text
         :param friend_number: number of friend
+        from Qt callback
         """
+        if not text:
+            return
         if friend_number is None:
             friend_number = self._contacts_manager.get_active_number()
-
+        if friend_number is None  or friend_number < 0:
-        if not text or friend_number < 0:
+            LOG.error(f"No _contacts_manager.get_active_number")
             return
+        assert_main_thread()
 
         friend = self._get_friend_by_number(friend_number)
+        if not friend:
+            LOG.error(f"No self._get_friend_by_number")
+            return
         messages = self._split_message(text.encode('utf-8'))
         t = util.get_unix_time()
         for message in messages:
@@ -106,7 +138,7 @@ class Messenger(tox_save.ToxSave):
                 message_id = self._tox.friend_send_message(friend_number, message.type, message.text.encode('utf-8'))
                 message.tox_message_id = message_id
         except Exception as ex:
-            util.log('Sending pending messages failed with ' + str(ex))
+            LOG.warn('Sending pending messages failed with ' + str(ex))
 
     # -----------------------------------------------------------------------------------------------------------------
     # Messaging - groups
@@ -141,7 +173,13 @@ class Messenger(tox_save.ToxSave):
         """
         t = util.get_unix_time()
         group = self._get_group_by_number(group_number)
+        if not group:
+            LOG.error(f"FixMe new_group_message _get_group_by_number({group_number})")
+            return
         peer = group.get_peer_by_id(peer_id)
+        if not peer:
+            LOG.error('FixMe new_group_message group.get_peer_by_id ' + str(peer_id))
+            return
         text_message = TextMessage(message, MessageAuthor(peer.name, MESSAGE_AUTHOR['GC_PEER']), t, message_type)
         self._add_message(text_message, group)
 
@@ -156,10 +194,17 @@ class Messenger(tox_save.ToxSave):
             group = self._get_group_by_public_key(group_peer_contact.group_pk)
             group_number = group.number
 
-        if not text or group_number < 0 or peer_id < 0:
+        if not text:
+            return
+        if group.number < 0:
+            return
+        if peer_id and peer_id < 0:
             return
         
+        assert_main_thread()
+        # FixMe: peer_id is None?
         group_peer_contact = self._contacts_manager.get_or_create_group_peer_contact(group_number, peer_id)
+        # group_peer_contact now may be None
         group = self._get_group_by_number(group_number)
         messages = self._split_message(text.encode('utf-8'))
         t = util.get_unix_time()
@@ -182,9 +227,15 @@ class Messenger(tox_save.ToxSave):
         t = util.get_unix_time()
         group = self._get_group_by_number(group_number)
         peer = group.get_peer_by_id(peer_id)
+        if not peer:
+            LOG.warn('FixMe new_group_private_message group.get_peer_by_id ' + str(peer_id))
+            return
         text_message = TextMessage(message, MessageAuthor(peer.name, MESSAGE_AUTHOR['GC_PEER']),
                                    t, message_type)
         group_peer_contact = self._contacts_manager.get_or_create_group_peer_contact(group_number, peer_id)
+        if not group_peer_contact:
+            LOG.warn('FixMe new_group_private_message group_peer_contact ' + str(peer_id))
+            return
         self._add_message(text_message, group_peer_contact)
 
     # -----------------------------------------------------------------------------------------------------------------
@@ -285,21 +336,28 @@ class Messenger(tox_save.ToxSave):
 
     def _add_info_message(self, friend_number, text):
         friend = self._get_friend_by_number(friend_number)
+        assert friend
         message = InfoMessage(text, util.get_unix_time())
         friend.append_message(message)
         if self._contacts_manager.is_friend_active(friend_number):
             self._create_info_message_item(message)
 
     def _create_info_message_item(self, message):
+        assert_main_thread()
         self._items_factory.create_message_item(message)
         self._screen.messages.scrollToBottom()
 
     def _add_message(self, text_message, contact):
+        assert_main_thread()
+        if not contact:
+            LOG.warn("_add_message null contact")
+            return
         if self._contacts_manager.is_contact_active(contact):  # add message to list
             self._create_message_item(text_message)
             self._screen.messages.scrollToBottom()
             self._contacts_manager.get_curr_contact().append_message(text_message)
         else:
+            LOG.debug("_add_message not is_contact_active(contact)")
             contact.inc_messages()
             contact.append_message(text_message)
             if not contact.visibility:

toxygen/middleware/callbacks.py

@@ -1,15 +1,48 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+import sys
+import os
+import threading
 from PyQt5 import QtGui
 from wrapper.toxcore_enums_and_consts import *
 from wrapper.toxav_enums import *
 from wrapper.tox import bin_to_string
 import utils.ui as util_ui
 import utils.util as util
-import cv2
-import numpy as np
 from middleware.threads import invoke_in_main_thread, execute
 from notifications.tray import tray_notification
 from notifications.sound import *
-import threading
+from datetime import datetime
+
+iMAX_INT32 = 4294967295
+# callbacks can be called in any thread so were being careful
+def LOG_ERROR(l): print('EROR< '+l)
+def LOG_WARN(l):  print('WARN< '+l)
+def LOG_INFO(l):
+    bIsVerbose = hasattr(__builtins__, 'app') and app.oArgs.loglevel <= 20-1
+    if bIsVerbose: print('INFO< '+l)
+def LOG_DEBUG(l):
+    bIsVerbose = hasattr(__builtins__, 'app') and app.oArgs.loglevel <= 10-1
+    if bIsVerbose: print('DBUG< '+l)
+def LOG_TRACE(l):
+    bIsVerbose = hasattr(__builtins__, 'app') and app.oArgs.loglevel < 10-1
+    pass # print('TRACE+ '+l)
+
+global aTIMES
+aTIMES=dict()
+def bTooSoon(key, sSlot, fSec=10.0):
+    # rate limiting
+    global aTIMES
+    if sSlot not in aTIMES:
+        aTIMES[sSlot] = dict()
+    OTIME = aTIMES[sSlot]
+    now = datetime.now()
+    if key not in OTIME:
+        OTIME[key] = now
+        return False
+    delta = now - OTIME[key]
+    OTIME[key] = now
+    if delta.total_seconds() < fSec: return True
+    return False
 
 # TODO: refactoring. Use contact provider instead of manager
 
@@ -17,15 +50,47 @@ import threading
 # Callbacks - current user
 # -----------------------------------------------------------------------------------------------------------------
 
+global iBYTES
+iBYTES=0
+def sProcBytes(sFile=None):
+    if sys.platform == 'win32': return ''
+    global iBYTES
+    if sFile is None:
+        pid = os.getpid()
+        sFile = f"/proc/{pid}/net/softnet_stat"
+    if os.path.exists(sFile):
+        total = 0
+        with open(sFile, 'r') as iFd:
+            for elt in iFd.readlines():
+                i = elt.find(' ')
+                p = int(elt[:i], 16)
+                total = total + p
+        if iBYTES == 0:
+            iBYTES = total
+            return ''
+        diff = total - iBYTES
+        s = f' {diff // 1024} Kbytes'
+    else:
+        s = ''
+    return s
 
 def self_connection_status(tox, profile):
     """
     Current user changed connection status (offline, TCP, UDP)
     """
+    sSlot = 'self connection status'
     def wrapped(tox_link, connection, user_data):
-        print('Connection status: ', str(connection))
+        key = f"connection {connection}"
+        if bTooSoon(key, sSlot, 10): return
+        s = sProcBytes()
+        try:
             status = tox.self_get_status() if connection != TOX_CONNECTION['NONE'] else None
+            if status:
+                LOG_DEBUG(f"self_connection_status: connection={connection} status={status}" +' '+s)
             invoke_in_main_thread(profile.set_status, status)
+        except Exception as e:
+            LOG_ERROR(f"self_connection_status: {e}")
+            pass
 
     return wrapped
 
@@ -36,13 +101,17 @@ def self_connection_status(tox, profile):
 
 
 def friend_status(contacts_manager, file_transfer_handler, profile, settings):
+    sSlot = 'friend status'
     def wrapped(tox, friend_number, new_status, user_data):
         """
         Check friend's status (none, busy, away)
         """
-        print("Friend's #{} status changed!".format(friend_number))
+        LOG_DEBUG(f"Friend's #{friend_number} status changed")
+        key = f"friend_number {friend_number}"
+        if bTooSoon(key, sSlot, 10): return
         friend = contacts_manager.get_friend_by_number(friend_number)
-        if friend.status is None and settings['sound_notifications'] and profile.status != TOX_USER_STATUS['BUSY']:
+        if friend.status is None and settings['sound_notifications'] and \
+          profile.status != TOX_USER_STATUS['BUSY']:
             sound_notification(SOUND_NOTIFICATION['FRIEND_CONNECTION_STATUS'])
         invoke_in_main_thread(friend.set_status, new_status)
 
@@ -61,7 +130,7 @@ def friend_connection_status(contacts_manager, profile, settings, plugin_loader,
         """
         Check friend's connection status (offline, udp, tcp)
         """
-        print("Friend #{} connection status: {}".format(friend_number, new_status))
+        LOG_DEBUG(f"Friend #{friend_number} connection status: {new_status}")
         friend = contacts_manager.get_friend_by_number(friend_number)
         if new_status == TOX_CONNECTION['NONE']:
             invoke_in_main_thread(friend.set_status, None)
@@ -79,29 +148,35 @@ def friend_connection_status(contacts_manager, profile, settings, plugin_loader,
 
 
 def friend_name(contacts_provider, messenger):
+    sSlot = 'friend_name'
     def wrapped(tox, friend_number, name, size, user_data):
         """
         Friend changed his name
         """
-        print('New name friend #' + str(friend_number))
+        key = f"friend_number={friend_number}"
+        if bTooSoon(key, sSlot, 60): return
         friend = contacts_provider.get_friend_by_number(friend_number)
         old_name = friend.name
         new_name = str(name, 'utf-8')
+        LOG_DEBUG(f"get_friend_by_number #{friend_number} {new_name}")
         invoke_in_main_thread(friend.set_name, new_name)
         invoke_in_main_thread(messenger.new_friend_name, friend, old_name, new_name)
 
     return wrapped
 
-
 def friend_status_message(contacts_manager, messenger):
+    sSlot = 'status_message'
     def wrapped(tox, friend_number, status_message, size, user_data):
         """
         :return: function for callback friend_status_message. It updates friend's status message
         and calls window repaint
         """
         friend = contacts_manager.get_friend_by_number(friend_number)
+        key = f"friend_number={friend_number}"
+        if bTooSoon(key, sSlot, 10): return
+
         invoke_in_main_thread(friend.set_status_message, str(status_message, 'utf-8'))
-        print('User #{} has new status message'.format(friend_number))
+        LOG_DEBUG(f'User #{friend_number} has new status message')
         invoke_in_main_thread(messenger.send_messages, friend_number)
 
     return wrapped
@@ -112,15 +187,19 @@ def friend_message(messenger, contacts_manager, profile, settings, window, tray)
         """
         New message from friend
         """
+        LOG_DEBUG(f"friend_message #{friend_number}")
         message = str(message, 'utf-8')
         invoke_in_main_thread(messenger.new_message, friend_number, message_type, message)
         if not window.isActiveWindow():
             friend = contacts_manager.get_friend_by_number(friend_number)
-            if settings['notifications'] and profile.status != TOX_USER_STATUS['BUSY'] and not settings.locked:
+            if settings['notifications'] \
+              and profile.status != TOX_USER_STATUS['BUSY'] \
+              and not settings.locked:
                 invoke_in_main_thread(tray_notification, friend.name, message, tray, window)
             if settings['sound_notifications'] and profile.status != TOX_USER_STATUS['BUSY']:
                 sound_notification(SOUND_NOTIFICATION['MESSAGE'])
             icon = os.path.join(util.get_images_directory(), 'icon_new_messages.png')
+            if tray:
                 invoke_in_main_thread(tray.setIcon, QtGui.QIcon(icon))
 
     return wrapped
@@ -131,7 +210,7 @@ def friend_request(contacts_manager):
         """
         Called when user get new friend request
         """
-        print('Friend request')
+        LOG_DEBUG(f'Friend request')
         key = ''.join(chr(x) for x in public_key[:TOX_PUBLIC_KEY_SIZE])
         tox_id = bin_to_string(key, TOX_PUBLIC_KEY_SIZE)
         invoke_in_main_thread(contacts_manager.process_friend_request, tox_id, str(message, 'utf-8'))
@@ -140,9 +219,12 @@ def friend_request(contacts_manager):
 
 
 def friend_typing(messenger):
+    sSlot = "friend_typing"
     def wrapped(tox, friend_number, typing, user_data):
+        key = f"friend_number={friend_number}"
+        if bTooSoon(key, sSlot, 10): return
+        LOG_DEBUG(f"friend_typing #{friend_number}")
         invoke_in_main_thread(messenger.friend_typing, friend_number, typing)
-
     return wrapped
 
 
@@ -164,7 +246,7 @@ def tox_file_recv(window, tray, profile, file_transfer_handler, contacts_manager
     """
     def wrapped(tox, friend_number, file_number, file_type, size, file_name, file_name_size, user_data):
         if file_type == TOX_FILE_KIND['DATA']:
-            print('File')
+            LOG_DEBUG(f'file_transfer_handler File')
             try:
                 file_name = str(file_name[:file_name_size], 'utf-8')
             except:
@@ -176,15 +258,18 @@ def tox_file_recv(window, tray, profile, file_transfer_handler, contacts_manager
                                   file_name)
             if not window.isActiveWindow():
                 friend = contacts_manager.get_friend_by_number(friend_number)
-                if settings['notifications'] and profile.status != TOX_USER_STATUS['BUSY'] and not settings.locked:
+                if settings['notifications'] \
+                  and profile.status != TOX_USER_STATUS['BUSY'] \
+                  and not settings.locked:
                     file_from = util_ui.tr("File from")
                     invoke_in_main_thread(tray_notification, file_from + ' ' + friend.name, file_name, tray, window)
                 if settings['sound_notifications'] and profile.status != TOX_USER_STATUS['BUSY']:
                     sound_notification(SOUND_NOTIFICATION['FILE_TRANSFER'])
+                if tray:
                     icon = util.join_path(util.get_images_directory(), 'icon_new_messages.png')
                     invoke_in_main_thread(tray.setIcon, QtGui.QIcon(icon))
         else:  # avatar
-            print('Avatar')
+            LOG_DEBUG(f'file_transfer_handler Avatar')
             invoke_in_main_thread(file_transfer_handler.incoming_avatar,
                                   friend_number,
                                   file_number,
@@ -259,15 +344,17 @@ def lossy_packet(plugin_loader):
 # -----------------------------------------------------------------------------------------------------------------
 
 def call_state(calls_manager):
-    def wrapped(toxav, friend_number, mask, user_data):
+    def wrapped(iToxav, friend_number, mask, user_data):
         """
         New call state
         """
-        print(friend_number, mask)
+        LOG_DEBUG(f"call_state #{friend_number}")
         if mask == TOXAV_FRIEND_CALL_STATE['FINISHED'] or mask == TOXAV_FRIEND_CALL_STATE['ERROR']:
             invoke_in_main_thread(calls_manager.stop_call, friend_number, True)
         else:
-            calls_manager.toxav_call_state_cb(friend_number, mask)
+            # guessing was calls_manager.
+            #? incoming_call
+            calls_manager._call.toxav_call_state_cb(friend_number, mask)
 
     return wrapped
 
@@ -277,7 +364,7 @@ def call(calls_manager):
         """
         Incoming call from friend
         """
-        print(friend_number, audio, video)
+        LOG_DEBUG(f"Incoming call from {friend_number} {audio} {video}")
         invoke_in_main_thread(calls_manager.incoming_call, audio, video, friend_number)
 
     return wrapped
@@ -288,7 +375,9 @@ def callback_audio(calls_manager):
         """
         New audio chunk
         """
-        calls_manager.call.audio_chunk(
+        LOG_DEBUG(f"callback_audio #{friend_number}")
+        # dunno was .call
+        calls_manager._call.audio_chunk(
             bytes(samples[:audio_samples_per_channel * 2 * audio_channels_count]),
             audio_channels_count,
             rate)
@@ -324,6 +413,9 @@ def video_receive_frame(toxav, friend_number, width, height, y, u, v, ystride, u
 
     It can be created from initial y, u, v using slices
     """
+    LOG_DEBUG(f"video_receive_frame from toxav_video_receive_frame_cb={friend_number}")
+    import cv2
+    import numpy as np
     try:
         y_size = abs(max(width, abs(ystride)))
         u_size = abs(max(width // 2, abs(ustride)))
@@ -349,7 +441,8 @@ def video_receive_frame(toxav, friend_number, width, height, y, u, v, ystride, u
 
         invoke_in_main_thread(cv2.imshow, str(friend_number), frame)
     except Exception as ex:
-        print(ex)
+        LOG_ERROR(f"video_receive_frame  {ex} #{friend_number}")
+        pass
 
 # -----------------------------------------------------------------------------------------------------------------
 # Callbacks - groups
@@ -361,16 +454,22 @@ def group_message(window, tray, tox, messenger, settings, profile):
     New message in group chat
     """
     def wrapped(tox_link, group_number, peer_id, message_type, message, length, user_data):
+        LOG_DEBUG(f"group_message #{group_number}")
         message = str(message[:length], 'utf-8')
         invoke_in_main_thread(messenger.new_group_message, group_number, message_type, message, peer_id)
         if window.isActiveWindow():
             return
         bl = settings['notify_all_gc'] or profile.name in message
         name = tox.group_peer_get_name(group_number, peer_id)
-        if settings['notifications'] and profile.status != TOX_USER_STATUS['BUSY'] and (not settings.locked) and bl:
+        if settings['sound_notifications'] and bl and \
-            invoke_in_main_thread(tray_notification, name, message, tray, window)
+           profile.status != TOX_USER_STATUS['BUSY']:
-        if settings['sound_notifications'] and bl and profile.status != TOX_USER_STATUS['BUSY']:
             sound_notification(SOUND_NOTIFICATION['MESSAGE'])
+        if False and settings['tray_icon'] and tray:
+            if settings['notifications'] and \
+               profile.status != TOX_USER_STATUS['BUSY'] and \
+                   (not settings.locked) and bl:
+                invoke_in_main_thread(tray_notification, name, message, tray, window)
+            if tray:
                 icon = util.join_path(util.get_images_directory(), 'icon_new_messages.png')
                 invoke_in_main_thread(tray.setIcon, QtGui.QIcon(icon))
 
@@ -382,35 +481,45 @@ def group_private_message(window, tray, tox, messenger, settings, profile):
     New private message in group chat
     """
     def wrapped(tox_link, group_number, peer_id, message_type, message, length, user_data):
+        LOG_DEBUG(f"group_private_message #{group_number}")
         message = str(message[:length], 'utf-8')
         invoke_in_main_thread(messenger.new_group_private_message, group_number, message_type, message, peer_id)
         if window.isActiveWindow():
             return
         bl = settings['notify_all_gc'] or profile.name in message
         name = tox.group_peer_get_name(group_number, peer_id)
-        if settings['notifications'] and profile.status != TOX_USER_STATUS['BUSY'] and (not settings.locked) and bl:
+        if settings['notifications'] and settings['tray_icon'] \
+           and profile.status != TOX_USER_STATUS['BUSY'] \
+           and (not settings.locked) and bl:
             invoke_in_main_thread(tray_notification, name, message, tray, window)
         if settings['sound_notifications'] and bl and profile.status != TOX_USER_STATUS['BUSY']:
             sound_notification(SOUND_NOTIFICATION['MESSAGE'])
         icon = util.join_path(util.get_images_directory(), 'icon_new_messages.png')
+        if tray and hasattr(tray, 'setIcon'):
             invoke_in_main_thread(tray.setIcon, QtGui.QIcon(icon))
 
     return wrapped
 
-
+# Exception ignored on calling ctypes callback function: <function group_invite.<locals>.wrapped at 0x7ffede910700>
 def group_invite(window, settings, tray, profile, groups_service, contacts_provider):
     def wrapped(tox, friend_number, invite_data, length, group_name, group_name_length, user_data):
+        LOG_DEBUG(f"group_invite friend_number={friend_number}")
         group_name = str(bytes(group_name[:group_name_length]), 'utf-8')
         invoke_in_main_thread(groups_service.process_group_invite,
                               friend_number, group_name,
                               bytes(invite_data[:length]))
         if window.isActiveWindow():
             return
-        if settings['notifications'] and profile.status != TOX_USER_STATUS['BUSY'] and not settings.locked:
+        bHasTray = tray and settings['tray_icon']
+        if settings['notifications'] \
+           and bHasTray \
+           and profile.status != TOX_USER_STATUS['BUSY'] \
+           and not settings.locked:
             friend = contacts_provider.get_friend_by_number(friend_number)
             title = util_ui.tr('New invite to group chat')
             text = util_ui.tr('{} invites you to group "{}"').format(friend.name, group_name)
             invoke_in_main_thread(tray_notification, title, text, tray, window)
+        if tray:
             icon = util.join_path(util.get_images_directory(), 'icon_new_messages.png')
             invoke_in_main_thread(tray.setIcon, QtGui.QIcon(icon))
 
@@ -418,7 +527,11 @@ def group_invite(window, settings, tray, profile, groups_service, contacts_provi
 
 
 def group_self_join(contacts_provider, contacts_manager, groups_service):
+    sSlot = 'group_self_join'
     def wrapped(tox, group_number, user_data):
+        key = f"group_number {group_number}"
+        if bTooSoon(key, sSlot, 10): return
+        LOG_DEBUG(f"group_self_join #{group_number}")
         group = contacts_provider.get_group_by_number(group_number)
         invoke_in_main_thread(group.set_status, TOX_USER_STATUS['NONE'])
         invoke_in_main_thread(groups_service.update_group_info, group)
@@ -428,8 +541,15 @@ def group_self_join(contacts_provider, contacts_manager, groups_service):
 
 
 def group_peer_join(contacts_provider, groups_service):
+    sSlot = "group_peer_join"
     def wrapped(tox, group_number, peer_id, user_data):
+        key = f"group_peer_join #{group_number} peer_id={peer_id}"
+        if bTooSoon(key, sSlot, 20): return
         group = contacts_provider.get_group_by_number(group_number)
+        if peer_id > group._peers_limit:
+            LOG_ERROR(key +f" {peer_id} > {group._peers_limit}")
+            return
+        LOG_DEBUG(key)
         group.add_peer(peer_id)
         invoke_in_main_thread(groups_service.generate_peers_list)
         invoke_in_main_thread(groups_service.update_group_info, group)
@@ -438,29 +558,49 @@ def group_peer_join(contacts_provider, groups_service):
 
 
 def group_peer_exit(contacts_provider, groups_service, contacts_manager):
-    def wrapped(tox, group_number, peer_id, message, length, user_data):
+    def wrapped(tox,
+                group_number, peer_id,
+                exit_type, name, name_length,
+                message, length,
+                user_data):
         group = contacts_provider.get_group_by_number(group_number)
+        if group:
+            LOG_DEBUG(f"group_peer_exit #{group_number} peer_id={peer_id} exit_type={exit_type}")
             group.remove_peer(peer_id)
             invoke_in_main_thread(groups_service.generate_peers_list)
+        else:
+            LOG_WARN(f"group_peer_exit group not found #{group_number} peer_id={peer_id}")
 
     return wrapped
 
-
 def group_peer_name(contacts_provider, groups_service):
     def wrapped(tox, group_number, peer_id, name, length, user_data):
+        LOG_DEBUG(f"group_peer_name #{group_number} peer_id={peer_id}")
         group = contacts_provider.get_group_by_number(group_number)
         peer = group.get_peer_by_id(peer_id)
+        if peer:
             peer.name = str(name[:length], 'utf-8')
             invoke_in_main_thread(groups_service.generate_peers_list)
+        else:
+            # FixMe: known signal to revalidate roles...
+            #_peers = [(p._name, p._peer_id) for p in group.get_peers()]
+            LOG_TRACE(f"remove_peer group {group!r} has no peer_id={peer_id} in _peers!r")
+            return
 
     return wrapped
 
 
 def group_peer_status(contacts_provider, groups_service):
     def wrapped(tox, group_number, peer_id, peer_status, user_data):
+        LOG_DEBUG(f"group_peer_status #{group_number} peer_id={peer_id}")
         group = contacts_provider.get_group_by_number(group_number)
         peer = group.get_peer_by_id(peer_id)
+        if peer:
             peer.status = peer_status
+        else:
+            # _peers = [(p._name, p._peer_id) for p in group.get_peers()]
+            LOG_TRACE(f"remove_peer group {group!r} has no peer_id={peer_id} in _peers!r")
+        # TODO: add info message
         invoke_in_main_thread(groups_service.generate_peers_list)
 
     return wrapped
@@ -468,32 +608,62 @@ def group_peer_status(contacts_provider, groups_service):
 
 def group_topic(contacts_provider):
     def wrapped(tox, group_number, peer_id, topic, length, user_data):
+        LOG_DEBUG(f"group_topic #{group_number} peer_id={peer_id}")
         group = contacts_provider.get_group_by_number(group_number)
+        if group:
             topic = str(topic[:length], 'utf-8')
             invoke_in_main_thread(group.set_status_message, topic)
+        else:
+            _peers = [(p._name, p._peer_id) for p in group.get_peers()]
+            LOG_WARN(f"group_topic {group!r} has no peer_id={peer_id} in {_peers!r}")
+        # TODO: add info message
 
     return wrapped
 
-
 def group_moderation(groups_service, contacts_provider, contacts_manager, messenger):
-
     def update_peer_role(group, mod_peer_id, peer_id, new_role):
         peer = group.get_peer_by_id(peer_id)
+        if peer:
             peer.role = new_role
             # TODO: add info message
+        else:
+            # FixMe: known signal to revalidate roles...
+            # _peers = [(p._name, p._peer_id) for p in group.get_peers()]
+            LOG_TRACE(f"update_peer_role group {group!r} has no peer_id={peer_id} in _peers!r")
+        # TODO: add info message
 
     def remove_peer(group, mod_peer_id, peer_id, is_ban):
+        peer = group.get_peer_by_id(peer_id)
+        if peer:
             contacts_manager.remove_group_peer_by_id(group, peer_id)
             group.remove_peer(peer_id)
+        else:
+            # FixMe: known signal to revalidate roles...
+            #_peers = [(p._name, p._peer_id) for p in group.get_peers()]
+            LOG_TRACE(f"remove_peer group {group!r} has no peer_id={peer_id} in _peers!r")
         # TODO: add info message
 
+    # source_peer_number, target_peer_number,
     def wrapped(tox, group_number, mod_peer_id, peer_id, event_type, user_data):
+        if mod_peer_id == iMAX_INT32 or peer_id == iMAX_INT32:
+            # FixMe: known signal to revalidate roles...
+            return
+        LOG_DEBUG(f"group_moderation #{group_number} mod_id={mod_peer_id} peer_id={peer_id} event_type={event_type}")
         group = contacts_provider.get_group_by_number(group_number)
+        mod_peer = group.get_peer_by_id(mod_peer_id)
+        if not mod_peer:
+            #_peers = [(p._name, p._peer_id) for p in group.get_peers()]
+            LOG_TRACE(f"remove_peer group {group!r} has no mod_peer_id={mod_peer_id} in _peers!r")
+            return
+        peer = group.get_peer_by_id(peer_id)
+        if not peer:
+            # FixMe: known signal to revalidate roles...
+            #_peers = [(p._name, p._peer_id) for p in group.get_peers()]
+            LOG_TRACE(f"remove_peer group {group!r} has no peer_id={peer_id} in _peers!r")
+            return
 
         if event_type == TOX_GROUP_MOD_EVENT['KICK']:
             remove_peer(group, mod_peer_id, peer_id, False)
-        elif event_type == TOX_GROUP_MOD_EVENT['BAN']:
-            remove_peer(group, mod_peer_id, peer_id, True)
         elif event_type == TOX_GROUP_MOD_EVENT['OBSERVER']:
             update_peer_role(group, mod_peer_id, peer_id, TOX_GROUP_ROLE['OBSERVER'])
         elif event_type == TOX_GROUP_MOD_EVENT['USER']:
@@ -509,6 +679,7 @@ def group_moderation(groups_service, contacts_provider, contacts_manager, messen
 def group_password(contacts_provider):
 
     def wrapped(tox_link, group_number, password, length, user_data):
+        LOG_DEBUG(f"group_password #{group_number}")
         password = str(password[:length], 'utf-8')
         group = contacts_provider.get_group_by_number(group_number)
         group.password = password
@@ -519,6 +690,7 @@ def group_password(contacts_provider):
 def group_peer_limit(contacts_provider):
 
     def wrapped(tox_link, group_number, peer_limit, user_data):
+        LOG_DEBUG(f"group_peer_limit #{group_number}")
         group = contacts_provider.get_group_by_number(group_number)
         group.peer_limit = peer_limit
 
@@ -528,6 +700,7 @@ def group_peer_limit(contacts_provider):
 def group_privacy_state(contacts_provider):
 
     def wrapped(tox_link, group_number, privacy_state, user_data):
+        LOG_DEBUG(f"group_privacy_state #{group_number}")
         group = contacts_provider.get_group_by_number(group_number)
         group.is_private = privacy_state == TOX_GROUP_PRIVACY_STATE['PRIVATE']
 
@@ -540,7 +713,7 @@ def group_privacy_state(contacts_provider):
 
 def init_callbacks(tox, profile, settings, plugin_loader, contacts_manager,
                    calls_manager, file_transfer_handler, main_window, tray, messenger, groups_service,
-                   contacts_provider):
+                   contacts_provider, ms=None):
     """
     Initialization of all callbacks.
     :param tox: Tox instance
@@ -557,6 +730,7 @@ def init_callbacks(tox, profile, settings, plugin_loader, contacts_manager,
     :param groups_service: GroupsService instance
     :param contacts_provider: ContactsProvider instance
     """
+
     # self callbacks
     tox.callback_self_connection_status(self_connection_status(tox, profile))
 

toxygen/middleware/threads.py

@@ -1,10 +1,32 @@
-from bootstrap.bootstrap import *
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
+import sys
 import threading
 import queue
-from utils import util
-import time
 from PyQt5 import QtCore
 
+from bootstrap.bootstrap import *
+from bootstrap.bootstrap import download_nodes_list
+from wrapper.toxcore_enums_and_consts import TOX_USER_STATUS, TOX_CONNECTION
+import wrapper_tests.support_testing as ts
+from utils import util
+
+import time
+sleep = time.sleep
+
+# LOG=util.log
+global LOG
+import logging
+LOG = logging.getLogger('app.'+'threads')
+# log = lambda x: LOG.info(x)
+
+def LOG_ERROR(l): print('EROR+ '+l)
+def LOG_WARN(l):  print('WARN+ '+l)
+def LOG_INFO(l):  print('INFO+ '+l)
+def LOG_DEBUG(l): print('DBUG+ '+l)
+def LOG_TRACE(l): pass # print('TRACE+ '+l)
+
+iLAST_CONN = 0
+iLAST_DELTA = 60
 
 # -----------------------------------------------------------------------------------------------------------------
 # Base threads
@@ -12,25 +34,45 @@ from PyQt5 import QtCore
 
 class BaseThread(threading.Thread):
 
-    def __init__(self):
+    def __init__(self, name=None, target=None):
-        super().__init__()
         self._stop_thread = False
+        if name:
+            super().__init__(name=name, target=target)
+        else:
+            super().__init__(target=target)
 
-    def stop_thread(self):
+    def stop_thread(self, timeout=-1):
         self._stop_thread = True
-        self.join()
+        if timeout < 0:
-
+            timeout = ts.iTHREAD_TIMEOUT
+        i = 0
+        while i < ts.iTHREAD_JOINS:
+            self.join(timeout)
+            if not self.is_alive(): break
+            i = i + 1
+        else:
+            LOG_WARN(f"BaseThread {self.name} BLOCKED after {ts.iTHREAD_JOINS}")
 
 class BaseQThread(QtCore.QThread):
 
-    def __init__(self):
+    def __init__(self, name=None):
+        # NO name=name
         super().__init__()
         self._stop_thread = False
+        self.name = str(id(self))
 
-    def stop_thread(self):
+    def stop_thread(self, timeout=-1):
         self._stop_thread = True
-        self.wait()
+        if timeout < 0:
-
+            timeout = ts.iTHREAD_TIMEOUT
+        i = 0
+        while i < ts.iTHREAD_JOINS:
+            self.wait(timeout)
+            if not self.isRunning(): break
+            i = i + 1
+            sleep(ts.iTHREAD_TIMEOUT)
+        else:
+            LOG_WARN(f"BaseQThread {self.name} BLOCKED")
 
 # -----------------------------------------------------------------------------------------------------------------
 # Toxcore threads
@@ -38,68 +80,88 @@ class BaseQThread(QtCore.QThread):
 
 class InitThread(BaseThread):
 
-    def __init__(self, tox, plugin_loader, settings, is_first_start):
+    def __init__(self, tox, plugin_loader, settings, app, is_first_start):
-        super().__init__()
+        super().__init__(name='InitThread')
-        self._tox, self._plugin_loader, self._settings = tox, plugin_loader, settings
+        self._tox = tox
+        self._plugin_loader = plugin_loader
+        self._settings = settings
+        self._app = app
         self._is_first_start = is_first_start
 
     def run(self):
+        # DBUG+ InitThread run: ERROR name 'ts' is not defined
+        import wrapper_tests.support_testing as ts
+        LOG_DEBUG('InitThread run: ')
+        try:
+            if self._is_first_start and ts.bAreWeConnected() and \
+              self._settings['download_nodes_list']:
+                LOG_INFO(f"downloading list of nodes {self._settings['download_nodes_list']}")
+                download_nodes_list(self._settings, oArgs=self._app._args)
+
+            if ts.bAreWeConnected():
+                LOG_INFO(f"calling test_net nodes")
+                self._app.test_net(oThread=self, iMax=4)
+
             if self._is_first_start:
-            # download list of nodes if needed
+                LOG_INFO('starting plugins')
-            download_nodes_list(self._settings)
-            # start plugins
                 self._plugin_loader.load()
 
-        # bootstrap
+        except Exception as e:
-        try:
+            LOG_DEBUG(f"InitThread run: ERROR {e}")
-            for data in generate_nodes():
-                if self._stop_thread:
-                    return
-                self._tox.bootstrap(*data)
-                self._tox.add_tcp_relay(*data)
-        except:
             pass
 
-        for _ in range(10):
+        for _ in range(ts.iTHREAD_JOINS):
             if self._stop_thread:
                 return
-            time.sleep(1)
+            sleep(ts.iTHREAD_SLEEP)
-
-        while not self._tox.self_get_connection_status():
-            try:
-                for data in generate_nodes(None):
-                    if self._stop_thread:
         return
-                    self._tox.bootstrap(*data)
-                    self._tox.add_tcp_relay(*data)
-            except:
-                pass
-            finally:
-                time.sleep(5)
-
 
 class ToxIterateThread(BaseQThread):
 
-    def __init__(self, tox):
+    def __init__(self, tox, app=None):
         super().__init__()
         self._tox = tox
+        self._app = app
         
     def run(self):
+        LOG_DEBUG('ToxIterateThread run: ')
         while not self._stop_thread:
+            try:
+                iMsec = self._tox.iteration_interval()
                 self._tox.iterate()
-            time.sleep(self._tox.iteration_interval() / 1000)
+            except Exception as e:
+                # Fatal Python error: Segmentation fault
+                LOG_ERROR(f"ToxIterateThread run: {e}")
+            else:
+                sleep(iMsec / 1000.0)
+                
+            global iLAST_CONN
+            if not iLAST_CONN:
+                iLAST_CONN = time.time()
+            # TRAC> TCP_common.c#203:read_TCP_packet recv buffer has 0 bytes, but requested 10 bytes
+
+            # and segv
+            if \
+                time.time() - iLAST_CONN > iLAST_DELTA and \
+                ts.bAreWeConnected() and \
+                self._tox.self_get_status() == TOX_USER_STATUS['NONE'] and \
+                self._tox.self_get_connection_status() == TOX_CONNECTION['NONE']:
+                    iLAST_CONN = time.time()
+                    LOG_INFO(f"ToxIterateThread calling test_net")
+                    invoke_in_main_thread(
+                        self._app.test_net, oThread=self, iMax=2)
                 
 
 class ToxAVIterateThread(BaseQThread):
-
     def __init__(self, toxav):
         super().__init__()
         self._toxav = toxav
 
     def run(self):
+        LOG_DEBUG('ToxAVIterateThread run: ')
         while not self._stop_thread:
             self._toxav.iterate()
-            time.sleep(self._toxav.iteration_interval() / 1000)
+            sleep(self._toxav.iteration_interval() / 1000)
 
 
 # -----------------------------------------------------------------------------------------------------------------
@@ -109,7 +171,7 @@ class ToxAVIterateThread(BaseQThread):
 class FileTransfersThread(BaseQThread):
 
     def __init__(self):
-        super().__init__()
+        super().__init__('FileTransfers')
         self._queue = queue.Queue()
         self._timeout = 0.01
 
@@ -124,14 +186,12 @@ class FileTransfersThread(BaseQThread):
             except queue.Empty:
                 pass
             except queue.Full:
-                util.log('Queue is full in _thread')
+                LOG_WARN('Queue is full in _thread')
             except Exception as ex:
-                util.log('Exception in _thread: ' + str(ex))
+                LOG_ERROR('in _thread: ' + str(ex))
 
 
 _thread = FileTransfersThread()
-
-
 def start_file_transfer_thread():
     _thread.start()
 

toxygen/middleware/tox_factory.py

@@ -1,27 +1,90 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 import user_data.settings
 import wrapper.tox
 import wrapper.toxcore_enums_and_consts as enums
 import ctypes
+import traceback
+import os
 
+global LOG
+import logging
+LOG = logging.getLogger('app.'+'tox_factory')
 
-def tox_factory(data=None, settings=None):
+from ctypes import *
+from utils import util
+from utils import ui as util_ui
+
+# callbacks can be called in any thread so were being careful
+# tox.py can be called by callbacks
+def LOG_ERROR(a): print('EROR> '+a)
+def LOG_WARN(a): print('WARN> '+a)
+def LOG_INFO(a):
+    bVERBOSE = hasattr(__builtins__, 'app') and app.oArgs.loglevel <= 20
+    if bVERBOSE: print('INFO> '+a)
+def LOG_DEBUG(a):
+    bVERBOSE = hasattr(__builtins__, 'app') and app.oArgs.loglevel <= 10
+    if bVERBOSE: print('DBUG> '+a)
+def LOG_TRACE(a):
+    bVERBOSE = hasattr(__builtins__, 'app') and app.oArgs.loglevel < 10
+    if bVERBOSE: print('TRAC> '+a)
+def LOG_LOG(a): print('TRAC> '+a)
+
+def tox_log_cb(iTox, level, file, line, func, message, *args):
+    """
+    * @param level The severity of the log message.
+    * @param file The source file from which the message originated.
+    * @param line The source line from which the message originated.
+    * @param func The function from which the message originated.
+    * @param message The log message.
+    * @param user_data The user data pointer passed to tox_new in options.
+    """
+    try:
+        file = str(file, 'UTF-8')
+        # root WARNING 3network.c#944:b'send_packet'attempted to send message with network family 10 (probably IPv6) on IPv4 socket
+        if file == 'network.c' and line in [944, 660]: return
+        func = str(func, 'UTF-8')
+        message = str(message, 'UTF-8')
+        message = f"{file}#{line}:{func} {message}"
+        LOG_LOG(message)
+    except Exception as e:
+        LOG_ERROR(f"tox_log_cb {e}")
+
+#tox_log_handler (context=0x24763d0, 
+#    level=LOGGER_LEVEL_TRACE, file=0x7fffe599fb99 "TCP_common.c", line=203, 
+#    func=0x7fffe599fc50 <__func__.2> "read_TCP_packet", 
+#    message=0x7fffba7fabd0 "recv buffer has 0 bytes, but requested 10 bytes", 
+#    userdata=0x0) at /var/local/src/c-toxcore/toxcore/tox.c:78
+        
+def tox_factory(data=None, settings=None, args=None, app=None):
     """
     :param data: user data from .tox file. None = no saved data, create new profile
     :param settings: current profile settings. None = default settings will be used
     :return: new tox instance
     """
-    if settings is None:
+    if not settings:
+        LOG_WARN("tox_factory using get_default_settings")
         settings = user_data.settings.Settings.get_default_settings()
+    else:
+        user_data.settings.clean_settings(settings)
 
+    try:
         tox_options = wrapper.tox.Tox.options_new()
+        tox_options.contents.ipv6_enabled = settings['ipv6_enabled']
         tox_options.contents.udp_enabled = settings['udp_enabled']
-    tox_options.contents.proxy_type = settings['proxy_type']
+        tox_options.contents.proxy_type = int(settings['proxy_type'])
-    tox_options.contents.proxy_host = bytes(settings['proxy_host'], 'UTF-8')
+        if type(settings['proxy_host']) == str:
-    tox_options.contents.proxy_port = settings['proxy_port']
+            tox_options.contents.proxy_host = bytes(settings['proxy_host'],'UTF-8')
+        elif type(settings['proxy_host']) == bytes:
+            tox_options.contents.proxy_host = settings['proxy_host']
+        else:
+            tox_options.contents.proxy_host = b''
+        tox_options.contents.proxy_port = int(settings['proxy_port'])
         tox_options.contents.start_port = settings['start_port']
         tox_options.contents.end_port = settings['end_port']
         tox_options.contents.tcp_port = settings['tcp_port']
-    tox_options.contents.local_discovery_enabled = settings['lan_discovery']
+        tox_options.contents.local_discovery_enabled = settings['local_discovery_enabled']
+        tox_options.contents.dht_announcements_enabled = settings['dht_announcements_enabled']
+        tox_options.contents.hole_punching_enabled = settings['hole_punching_enabled']
         if data:  # load existing profile
             tox_options.contents.savedata_type = enums.TOX_SAVEDATA_TYPE['TOX_SAVE']
             tox_options.contents.savedata_data = ctypes.c_char_p(data)
@@ -31,4 +94,32 @@ def tox_factory(data=None, settings=None):
             tox_options.contents.savedata_data = None
             tox_options.contents.savedata_length = 0
 
-    return wrapper.tox.Tox(tox_options)
+        # overrides
+        tox_options.contents.local_discovery_enabled = False
+        tox_options.contents.ipv6_enabled = False
+        tox_options.contents.hole_punching_enabled = False
+
+        LOG.debug("wrapper.tox.Tox settings: " +repr(settings))
+
+        if 'trace_enabled' in settings and settings['trace_enabled']:
+            LOG_INFO("settings['trace_enabled' disabled" )
+        elif tox_options._options_pointer:
+            c_callback = CFUNCTYPE(None, c_void_p, c_int, c_char_p, c_int, c_char_p, c_char_p, c_void_p)
+            tox_options.self_logger_cb = c_callback(tox_log_cb)
+            wrapper.tox.Tox.libtoxcore.tox_options_set_log_callback(
+                tox_options._options_pointer,
+                tox_options.self_logger_cb)
+        else:
+            LOG_WARN("No tox_options._options_pointer to add self_logger_cb" )
+
+        retval = wrapper.tox.Tox(tox_options)
+    except Exception as e:
+        if app and hasattr(app, '_log'):
+            pass
+        LOG_ERROR(f"wrapper.tox.Tox failed:  {e}")
+        LOG_WARN(traceback.format_exc())
+        raise
+
+    if app and hasattr(app, '_log'):
+        app._log("DEBUG: wrapper.tox.Tox succeeded")
+    return retval

toxygen/network/tox_dns.py

@@ -1,20 +1,40 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 import json
 import urllib.request
 import utils.util as util
 from PyQt5 import QtNetwork, QtCore
+try:
+    import requests
+except ImportError:
+    requests = None
 
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
 
 class ToxDns:
 
-    def __init__(self, settings):
+    def __init__(self, settings, log=None):
         self._settings = settings
+        self._log = log
 
     @staticmethod
     def _send_request(url, data):
+        if requests:
+            LOG.info('send_request loading with requests: ' + str(url))
+            headers = dict()
+            headers['Content-Type'] = 'application/json'
+            req = requests.get(url, headers=headers)
+            if req.status_code < 300:
+                retval = req.content
+            else:
+                raise LookupError(str(req.status_code))
+        else:
             req = urllib.request.Request(url)
             req.add_header('Content-Type', 'application/json')
             response = urllib.request.urlopen(req, bytes(json.dumps(data), 'utf-8'))
-        res = json.loads(str(response.read(), 'utf-8'))
+            retval = response.read()
+        res = json.loads(str(retval, 'utf-8'))
         if not res['c']:
             return res['tox_id']
         else:
@@ -29,12 +49,25 @@ class ToxDns:
         site = email.split('@')[1]
         data = {"action": 3, "name": "{}".format(email)}
         urls = ('https://{}/api'.format(site), 'http://{}/api'.format(site))
-        if not self._settings['proxy_type']:  # no proxy
+        if requests:
+            for url in urls:
+                LOG.info('TOX nodes loading with requests: ' + str(url))
+                try:
+                    headers = dict()
+                    headers['Content-Type'] = 'application/json'
+                    req = requests.get(url, headers=headers)
+                    if req.status_code < 300:
+                        result = req.content
+                        return result
+                except Exception as ex:
+                    LOG.error('ERROR: TOX DNS loading error with requests: ' + str(ex))
+
+        elif not self._settings['proxy_type']:  # no proxy
             for url in urls:
                 try:
                     return self._send_request(url, data)
                 except Exception as ex:
-                    util.log('TOX DNS ERROR: ' + str(ex))
+                    LOG.error('ERROR: TOX DNS ' + str(ex))
         else:  # proxy
             netman = QtNetwork.QNetworkAccessManager()
             proxy = QtNetwork.QNetworkProxy()
@@ -60,6 +93,6 @@ class ToxDns:
                     if not result['c']:
                         return result['tox_id']
                 except Exception as ex:
-                    util.log('TOX DNS ERROR: ' + str(ex))
+                    LOG.error('ERROR: TOX DNS ' + str(ex))
 
         return None  # error

toxygen/notifications/sound.py

@@ -3,6 +3,9 @@ import wave
 import pyaudio
 import os.path
 
+global LOG
+import logging
+LOG = logging.getLogger('app.'+__name__)
 
 SOUND_NOTIFICATION = {
     'MESSAGE': 0,
@@ -25,9 +28,19 @@ class AudioFile:
 
     def play(self):
         data = self.wf.readframes(self.chunk)
+        try:
             while data:
                 self.stream.write(data)
                 data = self.wf.readframes(self.chunk)
+        except Exception as e:
+            LOG.error(f"Error during AudioFile play {e!s}")
+            LOG.debug("Error during AudioFile play " \
+                      +' rate=' +str(self.wf.getframerate()) \
+                      + 'format=' +str(self.p.get_format_from_width(self.wf.getsampwidth())) \
+                      +' channels=' +str(self.wf.getnchannels()) \
+                      )
+
+            raise
 
     def close(self):
         self.stream.close()

toxygen/notifications/tray.py

@@ -10,7 +10,7 @@ def tray_notification(title, text, tray, window):
     :param tray: ref to tray icon
     :param window: main window
     """
-    if QtWidgets.QSystemTrayIcon.isSystemTrayAvailable():
+    if tray and QtWidgets.QSystemTrayIcon.isSystemTrayAvailable():
         if len(text) > 30:
             text = text[:27] + '...'
         tray.showMessage(title, text, QtWidgets.QSystemTrayIcon.NoIcon, 3000)

toxygen/plugin_support/plugin_support.py

@@ -1,3 +1,4 @@
+# -*- mode: python; indent-tabs-mode: nil; py-indent-offset: 4; coding: utf-8 -*-
 import utils.util as util
 import os
 import importlib
@@ -5,6 +6,14 @@ import inspect
 import plugins.plugin_super_class as pl
 import sys
 
+# LOG=util.log
+global LOG
+import logging
+LOG = logging.getLogger('plugin_support')
+def trace(msg, *args, **kwargs): LOG._log(0, msg, [])
+LOG.trace = trace
+
+log = lambda x: LOG.info(x)
 
 class Plugin:
 
@@ -46,38 +55,49 @@ class PluginLoader:
         """
         path = util.get_plugins_directory()
         if not os.path.exists(path):
-            util.log('Plugin dir not found')
+            self._app._LOG('WARN: Plugin directory not found: ' + path)
             return
-        else:
+
         sys.path.append(path)
         files = [f for f in os.listdir(path) if os.path.isfile(os.path.join(path, f))]
         for fl in files:
             if fl in ('plugin_super_class.py', '__init__.py') or not fl.endswith('.py'):
                 continue
-            name = fl[:-3]  # module name without .py
+            base_name = fl[:-3]  # module name without .py
             try:
-                module = importlib.import_module(name)  # import plugin
+                module = importlib.import_module(base_name)  # import plugin
-            except ImportError:
+                LOG.trace('Imported module: ' +base_name +' file: ' +fl)
-                util.log('Import error in module ' + name)
+            except ImportError as e:
+                LOG.warn(f"Import error:  {e}" +' file: ' +fl)
                 continue
             except Exception as ex:
-                util.log('Exception in module ' + name + ' Exception: ' + str(ex))
+                LOG.error('importing ' + base_name + ' Exception: ' + str(ex))
                 continue
             for elem in dir(module):
                 obj = getattr(module, elem)
                 # looking for plugin class in module
                 if not inspect.isclass(obj) or not hasattr(obj, 'is_plugin') or not obj.is_plugin:
                     continue
-                print('Plugin', elem)
                 try:  # create instance of plugin class
-                    instance = obj(self._app)
+                    instance = obj(self._app) # name, short_name, app
-                    is_active = instance.get_short_name() in self._settings['plugins']
+                    # needed by bday...
+                    instance._profile=self._app._ms._profile
+                    instance._settings=self._settings
+                    short_name = instance.get_short_name()
+                    is_active = short_name in self._settings['plugins']
                     if is_active:
+                        try:
                             instance.start()
+                            self._app.LOG('INFO: Started Plugin ' +short_name)
+                        except Exception as e:
+                            self._app.LOG.error(f"Starting Plugin ' +short_name +'  {e}")
+                    # else: LOG.info('Defined Plugin ' +short_name)
                 except Exception as ex:
-                    util.log('Exception in module ' + name + ' Exception: ' + str(ex))
+                    LOG.error('in module ' + short_name + ' Exception: ' + str(ex))
                     continue
-                self._plugins[instance.get_short_name()] = Plugin(instance, is_active)
+                short_name = instance.get_short_name()
+                self._plugins[short_name] = Plugin(instance, is_active)
+                LOG.info('Added plugin: ' +short_name +' from file: ' +fl)
                 break
 
     def callback_lossless(self, friend_number, data):
@@ -126,7 +146,13 @@ class PluginLoader:
         """
         Return window or None for specified plugin
         """
+        try:
+            if key in self._plugins and hasattr(self._plugins[key], 'instance'):
                 return self._plugins[key].instance.get_window()
+        except Exception as e:
+            self._app.LOG('WARN: ' +key +' _plugins no slot instance: ' +str(e))
+
+        return None
 
     def toggle_plugin(self, key):
         """
@@ -162,6 +188,7 @@ class PluginLoader:
         for plugin in self._plugins.values():
             if not plugin.is_active:
                 continue
+
             try:
                 result.extend(plugin.instance.get_menu(num))
             except:
@@ -173,6 +200,10 @@ class PluginLoader:
         for plugin in self._plugins.values():
             if not plugin.is_active:
                 continue
+            if not hasattr(plugin.instance, 'get_message_menu'):
+                name = plugin.instance.get_short_name()
+                self._app.LOG('WARN: get_message_menu not found: ' + name)
+                continue
             try:
                 result.extend(plugin.instance.get_message_menu(menu, selected_text))
             except:
@@ -189,6 +220,11 @@ class PluginLoader:
             del self._plugins[key]
 
     def reload(self):
-        print('Reloading plugins')
+        path = util.get_plugins_directory()
+        if not os.path.exists(path):
+            self._app.LOG('WARN: Plugin directory not found: ' + path)
+            return
+
         self.stop()
+        self._app.LOG('INFO: Reloading plugins from ' +path)
         self.load()