BUG: Memory Leak + processes are not stopped + m*Sql retry

[spirillen]

Description

Extremely high consumption of memory, See screenshot

MySql connection keeps breaking, there should be a retry 3 times (with delay), before failing. the reasons for a broken connection can be many.

Configuration

cli_testing:
  cidr_expand: true
  db_type: mariadb
  display_mode:
    all: true
    colour: true
    dots: false
    execution_time: true
    less: false
    percentage: true
    quiet: false
    simple: false
    status: all
  file_generation:
    analytic: true
    hosts: false
    merge_output_dirs: false
    no_file: false
    plain: true
    unified_results: false
  local_network: true
  sorting_mode:
    hierarchical: true
debug:
  active: true
  level: critical
lookup:
  dns: true
  http_status_code: true
  netinfo: true
  reputation: false
  special: true
  timeout: 5
  whois: true
user_agent:
  browser: firefox
  custom: null
  platform: linux
dns:
  follow_server_order: false
  trust_server: false
  protocol: UDP
  server:
    - 130.226.161.34
    - 89.233.43.71
    - 185.121.177.177
    - 163.172.168.171
    - 192.168.1.100:5302
    - 192.168.1.6
collection:
   push: true
   url_base: <snip>

Reproduction

Any bigger zone file to test

pyfunceble -w 60 --rpz -f /tmp/pyfunceble/rpz.mypdns.cloud

Expected behavior

No error and all pyfunceble threads to disappear.

Screenshots

Versions

OS: Ubuntu 20.04

Python Version: 3.9.5

PyFunceble Version: 4.0.2b1.dev

Additional context

[funilrys]

Hey @spirillen, I can't really reproduce nor tell where this comes from.
Is it happening on an early version of Python ? What about an early version of PyFunceble?

Cheers!

[spirillen]

Hey @funilrys no I'm chasing a wild goose here and says it python 3.9.5... 😉

Did you remember to enable the collection feature in your test?

[funilrys]

This one is probably of of process spawning. I can't really do anything about this :/

When a process if forked/spawn memory is cloned to. Therefore, the higher your amount workers are, the more is copied/isolated.