Sync last updater version

This commit is contained in:
DSR! 2024-12-03 11:15:01 -03:00
parent e61676e8cb
commit a79e755eca
11 changed files with 175 additions and 40 deletions

View File

@ -31,9 +31,10 @@ re_download = die_win64_portable_(?:\S+).zip
[Portmon] [Portmon]
folder = Monitor\Portmon folder = Monitor\Portmon
url = https://docs.microsoft.com/en-us/sysinternals/downloads/portmon url = https://raw.githubusercontent.com/MicrosoftDocs/sysinternals/main/sysinternals/downloads/portmon.md
update_url = https://download.sysinternals.com/files/PortMon.zip update_url = https://download.sysinternals.com/files/PortMon.zip
re_version = <h1 [^>]*>Portmon for Windows v(.*?)</h1> from = web
re_version = # Portmon v(\d+\.\d+)
``` ```
Los valores utilizados para la configuración son: Los valores utilizados para la configuración son:
@ -65,6 +66,27 @@ Combinando el uso de `update_url` y `re_download` se consiguen las siguientes es
Esto es útil para arreglar los links de descarga de GitHub o Sourceforge. Esto es útil para arreglar los links de descarga de GitHub o Sourceforge.
4. También se dispone de un método de detección de nuevas versiones que en lugar de regex usa las cabeceras http con las que responde el servidor. 4. También se dispone de un método de detección de nuevas versiones que en lugar de regex usa las cabeceras http con las que responde el servidor.
## Parámetros de Línea de Comandos
El actualizador ofrece un conjunto flexible de parámetros para controlar su comportamiento:
| Parámetro | Descripción |
|--------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|
| `-h, --help` | Muestra este mensaje de ayuda y finaliza. |
| `-v, --version` | Muestra el número de versión del programa y finaliza. |
| `-u [UPDATE ...], --update [UPDATE ...]` | Especifica una lista de herramientas a actualizar. Si no se proporciona, se actualizarán todas. |
| `-dsu, --disable-self-update` | Desactiva la auto-actualización automática del script. |
| `-dfc, --disable-folder-clean` | Evita limpiar la carpeta de herramientas durante las actualizaciones. |
| `-dr, --disable-repack` | Impide empaquetar nuevamente las herramientas después del proceso de actualización. |
| `-dic, --disable-install-check` | Omite la verificación de si las herramientas están instaladas correctamente. |
| `-dpb, --disable-progress-bar` | Desactiva la barra de progreso durante las descargas. |
| `-sft {full,version,name}, --save-format-type {full,version,name}` | Especifica el tipo de formato para guardar las actualizaciones comprimidas: `full`, `version` o `name`. |
| `-f, --force` | Fuerza la descarga de actualizaciones, incluso si ya están actualizadas. |
| `-uga USE_GITHUB_API, --use-github-api USE_GITHUB_API` | Usa la API de GitHub para actualizaciones, especificando el token para autenticarse. |
| `-udp, --update-default-params` | Actualiza los parámetros predeterminados almacenados en la configuración. |
| `-dmc, --disable-mutex-check` | Permite ejecutar múltiples instancias del script desactivando la verificación de mutex. |
| `-d, --debug` | Activa la salida detallada de depuración para resolver problemas. |
## Ejemplos ## Ejemplos
La herramienta soporta varios comandos y combinaciones. Estos son los mas usados. La herramienta soporta varios comandos y combinaciones. Estos son los mas usados.

View File

@ -31,9 +31,10 @@ re_download = die_win64_portable_(?:\S+).zip
[Portmon] [Portmon]
folder = Monitor\Portmon folder = Monitor\Portmon
url = https://docs.microsoft.com/en-us/sysinternals/downloads/portmon url = https://raw.githubusercontent.com/MicrosoftDocs/sysinternals/main/sysinternals/downloads/portmon.md
update_url = https://download.sysinternals.com/files/PortMon.zip update_url = https://download.sysinternals.com/files/PortMon.zip
re_version = <h1 [^>]*>Portmon for Windows v(.*?)</h1> from = web
re_version = # Portmon v(\d+\.\d+)
``` ```
The values used for configuration are: The values used for configuration are:
@ -63,6 +64,27 @@ Combining the use of `update_url` and `re_download` the following download strat
This is useful for fixing GitHub or Sourceforge download links. This is useful for fixing GitHub or Sourceforge download links.
4. A new version detection method is also available that instead of regex uses the http headers with which the server responds. 4. A new version detection method is also available that instead of regex uses the http headers with which the server responds.
## Command-line Parameters
The updater provides a flexible set of parameters to control its behavior:
| Parameter | Description |
|--------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|
| `-h, --help` | Show this help message and exit. |
| `-v, --version` | Display the program's version number and exit. |
| `-u [UPDATE ...], --update [UPDATE ...]` | Specify a list of tools to update. Defaults to updating all tools if not provided. |
| `-dsu, --disable-self-update` | Disable automatic self-update of this script. |
| `-dfc, --disable-folder-clean` | Skip cleaning the tool's folder during updates. |
| `-dr, --disable-repack` | Prevent repacking of tools after the update process. |
| `-dic, --disable-install-check` | Skip checking if the tools are properly installed. |
| `-dpb, --disable-progress-bar` | Disable the download progress bar for updates. |
| `-sft {full,version,name}, --save-format-type {full,version,name}` | Specify the save format type for compressed updates: `full`, `version`, or `name`. |
| `-f, --force` | Force the download of updates, even if they appear up to date. |
| `-uga USE_GITHUB_API, --use-github-api USE_GITHUB_API` | Use the GitHub API for updates, specifying the token to authenticate. |
| `-udp, --update-default-params` | Update the default parameters stored in the configuration. |
| `-dmc, --disable-mutex-check` | Allow multiple instances of the script to run simultaneously by disabling the mutex check. |
| `-d, --debug` | Enable detailed debug output for troubleshooting. |
## Examples ## Examples
The tool supports various commands and combinations. These are the most used. The tool supports various commands and combinations. These are the most used.

View File

@ -4,6 +4,9 @@ echo Update all Universal Update stuff and restart...
:: Kill updater process to be able to update all the files :: Kill updater process to be able to update all the files
taskkill /IM updater.exe /F taskkill /IM updater.exe /F
:: Clean updater mutex
del mutex.lock
:: Backup user tools.ini :: Backup user tools.ini
move tools.ini tools.ini.old move tools.ini tools.ini.old

View File

@ -4,6 +4,7 @@ import sys
import os import os
import colorama import colorama
import logging import logging
import psutil
from universal_updater.Updater import Updater from universal_updater.Updater import Updater
from universal_updater.ConfigManager import ConfigManager from universal_updater.ConfigManager import ConfigManager
@ -19,7 +20,8 @@ class UpdateManager:
""" """
Initialize the UpdateManager with a ConfigManager instance and command-line arguments. Initialize the UpdateManager with a ConfigManager instance and command-line arguments.
""" """
self.version = '2.2.0' self.version = '2.3.1'
self.process_mutex = 'mutex.lock'
self.config_file_name = 'tools.ini' self.config_file_name = 'tools.ini'
self.config_section_defaults = 'UpdaterConfig' self.config_section_defaults = 'UpdaterConfig'
self.config_section_self_update = 'UpdaterAutoUpdater' self.config_section_self_update = 'UpdaterAutoUpdater'
@ -43,16 +45,49 @@ class UpdateManager:
Version: {self.version} Version: {self.version}
""") """)
def exit_handler(self, signal, frame): def exit_handler(self, signum, frame):
""" """
Handles signals like SIGINT for graceful exit. Handles signals like SIGINT or SIGTERM for graceful exit.
:param signal: Signal type :param signum: Signal type (e.g., SIGINT, SIGTERM)
:param frame: Current stack frame :param frame: Current stack frame
""" """
print(colorama.Fore.YELLOW + 'SIGINT or CTRL-C detected. Exiting gracefully') signal_name = 'SIGINT' if signum == signal.SIGINT else 'SIGTERM'
print(colorama.Fore.YELLOW + f'{signal_name} detected. Exiting gracefully')
self.cleanup_mutex()
sys.exit(0) sys.exit(0)
def check_single_instance(self):
"""
Ensures only a single instance of the script is running by creating a lock file with the current PID.
If the lock file exists and contains a different PID, the script will exit.
"""
if self.arguments.disable_mutex_check:
print('Mutex check is disabled. Multiple instances can run concurrently.')
return
if os.path.exists(self.process_mutex):
with open(self.process_mutex, 'r') as lock:
existing_pid = int(lock.read().strip())
# Verify if the process with the PID exists
if existing_pid and psutil.pid_exists(existing_pid):
print(f"Another instance of the script is already running (PID: {existing_pid}). Exiting.")
sys.exit(1)
else:
print(f"Stale mutex detected (PID: {existing_pid}). Regenerating with current PID.")
# Create a new lock file with the current PID
with open(self.process_mutex, 'w') as lock:
lock.write(str(os.getpid()))
def cleanup_mutex(self):
"""
Removes the mutex file if mutex check is enabled and the file exists.
"""
if not self.arguments.disable_mutex_check and os.path.exists(self.process_mutex):
os.remove(self.process_mutex)
def get_argparse_default(self, option, default, is_bool=True): def get_argparse_default(self, option, default, is_bool=True):
""" """
Retrieves the default value for a given argparse option from the configuration. Retrieves the default value for a given argparse option from the configuration.
@ -83,54 +118,54 @@ class UpdateManager:
'-u', '-u',
'--update', '--update',
dest='update', dest='update',
help='list of tools (default: all)', help='Specify a list of tools to update. Defaults to updating all tools if not provided.',
nargs='*' nargs='*'
) )
parser.add_argument( parser.add_argument(
'-dsu', '-dsu',
'--disable-self-update', '--disable-self-update',
dest='disable_self_update', dest='disable_self_update',
help='disable self update of this script', help='Disable automatic self-update of this script.',
action=argparse.BooleanOptionalAction, action='store_true',
default=False default=False
) )
parser.add_argument( parser.add_argument(
'-dfc', '-dfc',
'--disable-folder-clean', '--disable-folder-clean',
dest='disable_clean', dest='disable_clean',
help='disable tool folder clean', help='Skip cleaning the tool\'s folder during updates.',
action=argparse.BooleanOptionalAction, action='store_true',
default=self.get_argparse_default('disable_clean', True) default=self.get_argparse_default('disable_clean', True)
) )
parser.add_argument( parser.add_argument(
'-dr', '-dr',
'--disable-repack', '--disable-repack',
dest='disable_repack', dest='disable_repack',
help='disable tool repack', help='Prevent repacking of tools after the update process.',
action=argparse.BooleanOptionalAction, action='store_true',
default=self.get_argparse_default('disable_repack', True) default=self.get_argparse_default('disable_repack', True)
) )
parser.add_argument( parser.add_argument(
'-dic', '-dic',
'--disable-install-check', '--disable-install-check',
dest='disable_install_check', dest='disable_install_check',
help='disable tool install check', help='Skip checking if the tools are properly installed.',
action=argparse.BooleanOptionalAction, action='store_true',
default=self.get_argparse_default('disable_install_check', False) default=self.get_argparse_default('disable_install_check', False)
) )
parser.add_argument( parser.add_argument(
'-dpb', '-dpb',
'--disable-progress-bar', '--disable-progress-bar',
dest='disable_progress', dest='disable_progress',
help='disable download progress bar', help='Disable the download progress bar for updates.',
action=argparse.BooleanOptionalAction, action='store_true',
default=self.get_argparse_default('disable_progress', False) default=self.get_argparse_default('disable_progress', False)
) )
parser.add_argument( parser.add_argument(
'-sft', '-sft',
'--save-format-type', '--save-format-type',
dest='save_format_type', dest='save_format_type',
help='compress save format name', help='Specify the save format type for compressed updates: "full", "version", or "name".',
choices=['full', 'version', 'name'], choices=['full', 'version', 'name'],
default=self.get_argparse_default('save_format_type', 'full', False) default=self.get_argparse_default('save_format_type', 'full', False)
) )
@ -138,31 +173,39 @@ class UpdateManager:
'-f', '-f',
'--force', '--force',
dest='force_download', dest='force_download',
help='force download', help='Force the download of updates, even if they appear up to date.',
action=argparse.BooleanOptionalAction, action='store_true',
default=False default=False
) )
parser.add_argument( parser.add_argument(
'-uga', '-uga',
'--use-github-api', '--use-github-api',
dest='use_github_api', dest='use_github_api',
help='use github api with this token', help='Use the GitHub API for updates, specifying the token to authenticate.',
default=self.get_argparse_default('use_github_api', '', False) default=self.get_argparse_default('use_github_api', '', False)
) )
parser.add_argument( parser.add_argument(
'-udp', '-udp',
'--update-default-params', '--update-default-params',
dest='update_default_params', dest='update_default_params',
help='update default params', help='Update the default parameters stored in the configuration.',
action=argparse.BooleanOptionalAction, action='store_true',
default=False
)
parser.add_argument(
'-dmc',
'--disable-mutex-check',
dest='disable_mutex_check',
help='Allow multiple instances of the script to run simultaneously by disabling the mutex check.',
action='store_true',
default=False default=False
) )
parser.add_argument( parser.add_argument(
'-d', '-d',
'--debug', '--debug',
dest='debug', dest='debug',
help='enable debug output', help='Enable detailed debug output for troubleshooting.',
action=argparse.BooleanOptionalAction, action='store_true',
default=False default=False
) )
@ -249,7 +292,7 @@ class UpdateManager:
try: try:
updater.update(self.config_section_self_update) updater.update(self.config_section_self_update)
except Exception as exception: except Exception as exception:
logging.info(exception) logging.error(exception)
# add missing new line separator # add missing new line separator
logging.info("\n") logging.info("\n")
@ -269,7 +312,8 @@ class UpdateManager:
try: try:
updater.update(tool) updater.update(tool)
except Exception as exception: except Exception as exception:
logging.info(exception) failed_updates += 1
logging.error(exception)
logging.info(colorama.Fore.YELLOW + f"\n[*] Update process completed: {total_updates - failed_updates} succeeded, {failed_updates} failed out of {total_updates} total updates.") logging.info(colorama.Fore.YELLOW + f"\n[*] Update process completed: {total_updates - failed_updates} succeeded, {failed_updates} failed out of {total_updates} total updates.")
@ -293,13 +337,19 @@ class UpdateManager:
""" """
Main entry point for the UpdateManager. Main entry point for the UpdateManager.
""" """
# setup signals
signal.signal(signal.SIGINT, self.exit_handler) signal.signal(signal.SIGINT, self.exit_handler)
signal.signal(signal.SIGTERM, self.exit_handler)
# setup script
self.change_current_directory() self.change_current_directory()
self.print_banner() self.print_banner()
self.parse_arguments() self.parse_arguments()
self.set_logging_level() self.set_logging_level()
self.check_single_instance()
self.update_default_params() self.update_default_params()
self.handle_updates() self.handle_updates()
self.cleanup_mutex()
# Entry point for the script # Entry point for the script

View File

@ -3,3 +3,4 @@ colorama>=0.4.4
tqdm>=4.62.3 tqdm>=4.62.3
py7zr>=0.16.1 py7zr>=0.16.1
rarfile>=4.0 rarfile>=4.0
psutil>=6.1.0

View File

@ -62,6 +62,7 @@ class Packer:
""" """
# download first "UnRAR for Windows" from https://www.rarlab.com/rar_add.htm # download first "UnRAR for Windows" from https://www.rarlab.com/rar_add.htm
# direct link: https://www.rarlab.com/rar/unrarw32.exe # direct link: https://www.rarlab.com/rar/unrarw32.exe
# new link: https://www.win-rar.com/predownload.html?&Version=32bit&L=0.
with rarfile.RarFile(file_path, 'r') as compressed: with rarfile.RarFile(file_path, 'r') as compressed:
compressed.extractall(unpack_path, pwd=file_pass) compressed.extractall(unpack_path, pwd=file_pass)

View File

@ -65,7 +65,11 @@ class Scraper:
""" """
Scrape web for version and download URL based on tool_config. Scrape web for version and download URL based on tool_config.
:return: Dictionary containing 'download_version' and 'download_url' :return: dict|bool: A dictionary containing:
- 'download_version' (str): Extracted version using regex.
- 'download_url' (str): Extracted or generated download URL.
Returns False if the version cannot be extracted.
:raises Exception: If required configuration fields are missing or HTTP requests fail.
""" """
update_url = self.tool_config.get('update_url', None) update_url = self.tool_config.get('update_url', None)
re_version = self.tool_config.get('re_version', None) re_version = self.tool_config.get('re_version', None)
@ -78,6 +82,9 @@ class Scraper:
# regex shit # regex shit
download_version = self.check_version_from_web(html_response.text, re_version) download_version = self.check_version_from_web(html_response.text, re_version)
if download_version is None:
return False
download_url = self.get_download_url_from_web(html_response.text, url, update_url, re_download) download_url = self.get_download_url_from_web(html_response.text, url, update_url, re_download)
logging.debug(f'{self.tool_name}: Regex matching done.') logging.debug(f'{self.tool_name}: Regex matching done.')
@ -90,7 +97,11 @@ class Scraper:
""" """
Scrape GitHub for version and download URL based on tool_config. Scrape GitHub for version and download URL based on tool_config.
:return: Dictionary containing 'download_version' and 'download_url' :return: dict|bool: A dictionary containing:
- 'download_version' (str): Extracted version from GitHub.
- 'download_url' (str): The determined or generated download URL.
Returns False if the version cannot be extracted.
:raises Exception: If required configuration fields are missing or HTTP requests fail.
""" """
if self.use_github_api: if self.use_github_api:
return self.scrape_github_api() return self.scrape_github_api()
@ -103,6 +114,9 @@ class Scraper:
logging.debug(f'{self.tool_name}: Version HTML fetched, starting regex matching for version.') logging.debug(f'{self.tool_name}: Version HTML fetched, starting regex matching for version.')
download_version = self.check_version_from_web(version_html_response.text, self.re_github_version) download_version = self.check_version_from_web(version_html_response.text, self.re_github_version)
if download_version is None:
return False
logging.debug(f'{self.tool_name}: Regex matching for version done.') logging.debug(f'{self.tool_name}: Regex matching for version done.')
# load second html # load second html
@ -124,7 +138,11 @@ class Scraper:
""" """
Scrape GitHub API for version and download URL based on tool_config. Scrape GitHub API for version and download URL based on tool_config.
:return: Dictionary containing 'download_version' and 'download_url' :return: dict|bool: A dictionary containing:
- 'download_version' (str): Extracted version from the API response.
- 'download_url' (str): The determined or generated download URL.
Returns False if the version cannot be extracted.
:raises Exception: If the API request fails or the response is invalid.
""" """
logging.debug(f'{self.tool_name}: Consuming GitHub via Api') logging.debug(f'{self.tool_name}: Consuming GitHub via Api')
github_repo = self.tool_config.get('url', None) github_repo = self.tool_config.get('url', None)
@ -142,6 +160,9 @@ class Scraper:
update_url = self.get_download_url_from_github(json_response) update_url = self.get_download_url_from_github(json_response)
download_version = self.check_version_from_github(json_response) download_version = self.check_version_from_github(json_response)
if download_version is None:
return False
logging.debug(f'{self.tool_name}: Version and download URL extracted.') logging.debug(f'{self.tool_name}: Version and download URL extracted.')
# regex shit # regex shit
@ -154,7 +175,11 @@ class Scraper:
""" """
Scrape HTTP headers for version based on tool_config. Scrape HTTP headers for version based on tool_config.
:return: Dictionary containing 'download_version' and 'download_url' :return: dict|bool: A dictionary containing:
- 'download_version' (str): Extracted version from the headers.
- 'download_url' (str): The update URL.
Returns False if the version cannot be extracted.
:raises Exception: If 'update_url' is missing or an HTTP error occurs.
""" """
# get http response # get http response
update_url = self.tool_config.get('update_url', None) update_url = self.tool_config.get('update_url', None)
@ -171,6 +196,9 @@ class Scraper:
raise Exception(colorama.Fore.RED + f'{self.tool_name}: Error {exception}') raise Exception(colorama.Fore.RED + f'{self.tool_name}: Error {exception}')
download_version = self.check_version_from_http(http_response.headers) download_version = self.check_version_from_http(http_response.headers)
if download_version is None:
return False
logging.debug(f'{self.tool_name}: Version extracted.') logging.debug(f'{self.tool_name}: Version extracted.')
return { return {
@ -196,7 +224,8 @@ class Scraper:
raise Exception(colorama.Fore.RED + f'{self.tool_name}: re_version regex not match ({re_version})') raise Exception(colorama.Fore.RED + f'{self.tool_name}: re_version regex not match ({re_version})')
if not self.force_download and local_version == html_regex_version[0]: if not self.force_download and local_version == html_regex_version[0]:
raise Exception(f'{self.tool_name}: {local_version} is the latest version') logging.info(f'{self.tool_name}: {local_version} is the latest version')
return None
logging.info(f'{self.tool_name}: updated from {local_version} --> {html_regex_version[0]}') logging.info(f'{self.tool_name}: updated from {local_version} --> {html_regex_version[0]}')
@ -225,7 +254,8 @@ class Scraper:
f'{self.tool_name}: no header is found with which to determine if there is an update') f'{self.tool_name}: no header is found with which to determine if there is an update')
if not self.force_download and local_version == remote_version: if not self.force_download and local_version == remote_version:
raise Exception(f'{self.tool_name}: {local_version} is the latest version') logging.info(f'{self.tool_name}: {local_version} is the latest version')
return None
logging.info(f'{self.tool_name}: updated from {local_version} --> {remote_version}') logging.info(f'{self.tool_name}: updated from {local_version} --> {remote_version}')
@ -241,14 +271,15 @@ class Scraper:
local_version = self.tool_config.get('local_version', '0') local_version = self.tool_config.get('local_version', '0')
if not self.force_download and local_version == json['tag_name']: if not self.force_download and local_version == json['tag_name']:
raise Exception(f'{self.tool_name}: {local_version} is the latest version') logging.info(f'{self.tool_name}: {local_version} is the latest version')
return None
logging.info(f'{self.tool_name}: updated from {local_version} --> {json["tag_name"]}') logging.info(f'{self.tool_name}: updated from {local_version} --> {json["tag_name"]}')
return json['tag_name'] return json['tag_name']
################# #################
# Check methods # Download url methods
################# #################
def get_download_url_from_web(self, html, html_url, update_url, re_download): def get_download_url_from_web(self, html, html_url, update_url, re_download):
""" """

View File

@ -141,6 +141,8 @@ class Updater:
Perform the update process for a given tool. Perform the update process for a given tool.
:param tool_name: Name of the tool to update :param tool_name: Name of the tool to update
:return: bool: True if the update completes successfully, False if no update is needed.
:raises Exception: If any step in the update process fails.
""" """
# tool data setup # tool data setup
self.tool_name = tool_name self.tool_name = tool_name
@ -155,6 +157,8 @@ class Updater:
# generate version and download data # generate version and download data
scrape_data = self.scraper.scrape_step() scrape_data = self.scraper.scrape_step()
if scrape_data is False:
return False
# download and process file # download and process file
update_file_path = self.download_step(scrape_data['download_url']) update_file_path = self.download_step(scrape_data['download_url'])
@ -165,3 +169,4 @@ class Updater:
self.post_update(processing_info) self.post_update(processing_info)
logging.info(f'{self.tool_name}: update complete') logging.info(f'{self.tool_name}: update complete')
return True

View File

@ -39,7 +39,7 @@ folder = ..\..\toolkit\Analysis\PE-Bear
url = hasherezade/pe-bear url = hasherezade/pe-bear
from = github from = github
local_version = v0.6.7.3 local_version = v0.6.7.3
re_download = PE-bear_(?:\S+)_x64_win_vs13.zip re_download = PE-bear_(?:\S+)_qt5_x86_win_vs19.zip
[PEStudio] [PEStudio]
folder = ..\..\toolkit\Analysis\PEStudio folder = ..\..\toolkit\Analysis\PEStudio
@ -521,7 +521,7 @@ folder = ..\..\bin\updater
url = indetectables-net/toolkit-updater url = indetectables-net/toolkit-updater
update_url = https://github.com/indetectables-net/toolkit-updater/archive/refs/heads/main.zip update_url = https://github.com/indetectables-net/toolkit-updater/archive/refs/heads/main.zip
from = github from = github
local_version = 2024.7.1 local_version = 2024.7.3
post_unpack = scripts\Toolkit-Updater.bat post_unpack = scripts\Toolkit-Updater.bat
disable_repack = True disable_repack = True

Binary file not shown.

Binary file not shown.