MPMDManager

An abstraction layer built on top of MPI that simplifies development of parallel programs in MPMD (Multiple Program Multiple Data) environments by providing intuitive functions for program identification, communication, and coordination.

MPI Abstraction

Complete lifecycle management for parallel programs with MPI abstraction layers.

Smart Connections

Reliable communication with configurable retry and error resilience.

Language Parity

Identical C/Python APIs with seamless cross-language compatibility.

Github

MPMDManager

Preview License

💡 Proposal
🛠️ Technologies
🚀 Getting Started
⚙️ Features
📚 API Reference
📜 Certification
📝 License
👨‍💻 Authors

💡 Proposal

MPMDManager provides two powerful abstractions for parallel programming:

Single World: Designed for programs launched together in MPI's MPMD mode (using colon-separated commands) where all processes share the same MPI_COMM_WORLD.
Multi World: For independently launched programs, each with their own MPI_COMM_WORLD, using MPI's client/server model with an ompi-server intermediary.

Available in both C (using native MPI) and Python (via mpi4py), these abstractions maintain identical functionality across languages while accommodating different developer preferences and use cases.

🛠️ Technologies

Built with:

C
- MPI (OpenMPI)
Python
- mpi4py

🚀 Getting Started

Prerequisites

MPI implementation (OpenMPI recommended)
C compiler (GCC recommended)
Python and mpi4py
Git

Installation

# Clone the repository
$ git clone https://github.com/RiadOliveira/MPMDManager.git

# Navigate to project directory
$ cd MPMDManager

Project Structure

The repository is organized as follows:

src/
├── single_world/
│   ├── c/
│   │   ├── abstraction/    # C implementation of Single World
│   │   ├── example/        # Example usage in C
│   │   └── run_example.sh  # Compiles and runs C example
│   └── python/
│       ├── abstraction/    # Python implementation
│       ├── example/        # Python example
│       └── run_example.sh  # Runs Python example
└── multi_world/
    ├── c/
    │   ├── abstraction/    # C implementation of Multi World
    │   ├── example/        # Client and server example
    │   └── run_example.sh  # Compiles and runs examples
    └── python/
        ├── abstraction/    # Python implementation
        ├── example/        # Python example
        └── run_example.sh  # Runs Python examples

Running Examples

To execute any example, navigate to the corresponding implementation directory:

cd src/<abstraction>/<language>/
./run_example.sh

Where:

<abstraction> is either single_world or multi_world.
<language> is either c or python.

Each run_example.sh script handles all necessary setup including:

Automatic compilation (for C implementations).
MPI environment configuration.
Clean execution.
Proper cleanup of temporary files.

⚙️ Features

Single World

Retrieve local program information (name, communicator, size).
Access process-specific details (rank).
Obtain remote program data by name or index (communicator, size).
Simplified inter-program communication setup.

Multi World (Client)

Connect to named servers with customizable retry logic by specifying:
- Max Attempts.
- Initial Wait Time (in milliseconds).
- Wait Increment Time (in milliseconds, added to the wait time for each attempt).
- Max Wait Time (in milliseconds).
Manage multiple server connections simultaneously.
Retrieve server communicators by name or index.
Disconnection from individual or all servers.

Multi World (Server)

Server lifecycle management (open/close).
Client connection approval with optional naming.
Client communicator retrieval by name or index.
Selective or complete client disconnection.
Server naming and identification.

📚 API Reference

Notes

All functions terminate with MPI_Abort() on failure.
Connection IDs can be names (strings) or indices (integers).
Python implementations use static methods for direct class access.

Single World

Function	Description	C Syntax	Python Syntax
Init	Initializes manager with program arguments	`const MPMDManager* Manager_Init(char** argv)`	`MPMDManager.init() -> None`
Finalize	Cleans up manager resources	`void Manager_Finalize(const MPMDManager*)`	`MPMDManager.finalize() -> None`
Local Name	Gets local program name	`const char* Manager_Local_name()`	`MPMDManager.local_name() -> str`
Local Comm	Gets local communicator	`const MPI_Comm* Manager_Local_comm()`	`MPMDManager.local_comm() -> MPI.Intracomm`
Local Rank	Gets process rank	`int Manager_Local_rank()`	`MPMDManager.local_rank() -> int`
Local Size	Gets program size	`unsigned int Manager_Local_size()`	`MPMDManager.local_size() -> int`
Comm To	Gets remote communicator	`const MPI_Comm* Manager_Comm_to(ConnectionId, IdType)`	`MPMDManager.comm_to(identifier: str \| int) -> MPI.Comm`
Size Of	Gets remote program size	`unsigned int Manager_Size_of(ConnectionId, IdType)`	`MPMDManager.size_of(identifier: str \| int) -> int`

Multi World (Client)

Function	Description	C Syntax	Python Syntax
Init	Initializes client manager	`const ClientManager* Client_Init(uint maxServers)`	`ClientManager.init() -> None`
Finalize	Cleans up client resources	`void Client_Finalize(const ClientManager*)`	`ClientManager.finalize() -> None`
Connect	Connects to named server with retry logic	`const MPI_Comm* Client_Connect(const char, const ConnectAttemptData)`	`ClientManager.connect(server_name: str, attemptData=ConnectAttemptData()) -> MPI.Intercomm`
Retrieve Server Comm	Gets server communicator	`const MPI_Comm* Client_Retrieve_Server_comm(ConnectionId, IdType)`	`ClientManager.retrieve_server_comm(server_id: str \| int) -> MPI.Intercomm`
Disconnect Server	Disconnects from specific server	`void Client_Disconnect_server(ConnectionId, IdType)`	`ClientManager.disconnect_server(server_id: str \| int) -> None`
Disconnect Servers	Disconnects from all servers	`void Client_Disconnect_servers()`	`ClientManager.disconnect_servers() -> None`

Multi World (Server)

Function	Description	C Syntax	Python Syntax
Init	Initializes server manager	`const ServerManager* Server_Init(char*, const char, uint)`	`ServerManager.init(server_name: str \| None = None) -> None`
Finalize	Cleans up server resources	`void Server_Finalize(const ServerManager*)`	`ServerManager.finalize() -> None`
Local Name	Gets server name	`const char* Server_Local_name()`	`ServerManager.local_name() -> str`
Open	Opens server for connections	`void Server_Open()`	`ServerManager.open() -> None`
Close	Closes server	`void Server_Close()`	`ServerManager.close() -> None`
Accept	Accepts client connection	`const MPI_Comm* Server_Accept(const char*)`	`ServerManager.accept(client_id: str \| None = None) -> Intercomm`
Retrieve Client Comm	Gets client communicator	`const MPI_Comm* Server_Retrieve_Client_comm(ConnectionId, IdType)`	`ServerManager.retrieve_client_comm(client_id: str \| int) -> MPI.Intercomm`
Disconnect Client	Disconnects specific client	`void Server_Disconnect_client(ConnectionId, IdType)`	`ServerManager.disconnect_client(client_id: str \| int) -> None`
Disconnect Clients	Disconnects all clients	`void Server_Disconnect_clients()`	`ServerManager.disconnect_clients() -> None`

📜 Certification

This software is officially registered with the Brazilian National Institute of Industrial Property (INPI) as a Computer Program:

Certificate Type: Certificate of Registration of a Computer Program
Registration Number: BR512025000477-0
Issuance Date: February 2025
Expiration Date: January 2076
Certificate URL: MPMDManager Certificate
Issuing Authority: INPI Brazil

📝 License

This project is MIT Licensed. See LICENSE file for more details.

👨‍💻 Authors

Ríad Oliveira Paulo Henrique

MPMDManager

Preview License

💡 Proposal
🛠️ Technologies
🚀 Getting Started
⚙️ Features
📚 API Reference
📜 Certification
📝 License
👨‍💻 Authors

💡 Proposal

MPMDManager provides two powerful abstractions for parallel programming:

Single World: Designed for programs launched together in MPI's MPMD mode (using colon-separated commands) where all processes share the same MPI_COMM_WORLD.
Multi World: For independently launched programs, each with their own MPI_COMM_WORLD, using MPI's client/server model with an ompi-server intermediary.

🛠️ Technologies

Built with:

C
- MPI (OpenMPI)
Python
- mpi4py

🚀 Getting Started

Prerequisites

MPI implementation (OpenMPI recommended)
C compiler (GCC recommended)
Python and mpi4py
Git

Installation

# Clone the repository
$ git clone https://github.com/RiadOliveira/MPMDManager.git

# Navigate to project directory
$ cd MPMDManager

Project Structure

The repository is organized as follows:

src/
├── single_world/
│   ├── c/
│   │   ├── abstraction/    # C implementation of Single World
│   │   ├── example/        # Example usage in C
│   │   └── run_example.sh  # Compiles and runs C example
│   └── python/
│       ├── abstraction/    # Python implementation
│       ├── example/        # Python example
│       └── run_example.sh  # Runs Python example
└── multi_world/
    ├── c/
    │   ├── abstraction/    # C implementation of Multi World
    │   ├── example/        # Client and server example
    │   └── run_example.sh  # Compiles and runs examples
    └── python/
        ├── abstraction/    # Python implementation
        ├── example/        # Python example
        └── run_example.sh  # Runs Python examples

Running Examples

To execute any example, navigate to the corresponding implementation directory:

cd src/<abstraction>/<language>/
./run_example.sh

Where:

<abstraction> is either single_world or multi_world.
<language> is either c or python.

Each run_example.sh script handles all necessary setup including:

Automatic compilation (for C implementations).
MPI environment configuration.
Clean execution.
Proper cleanup of temporary files.

⚙️ Features

Single World

Retrieve local program information (name, communicator, size).
Access process-specific details (rank).
Obtain remote program data by name or index (communicator, size).
Simplified inter-program communication setup.

Multi World (Client)

Connect to named servers with customizable retry logic by specifying:
- Max Attempts.
- Initial Wait Time (in milliseconds).
- Wait Increment Time (in milliseconds, added to the wait time for each attempt).
- Max Wait Time (in milliseconds).
Manage multiple server connections simultaneously.
Retrieve server communicators by name or index.
Disconnection from individual or all servers.

Multi World (Server)

Server lifecycle management (open/close).
Client connection approval with optional naming.
Client communicator retrieval by name or index.
Selective or complete client disconnection.
Server naming and identification.

📚 API Reference

Notes

All functions terminate with MPI_Abort() on failure.
Connection IDs can be names (strings) or indices (integers).
Python implementations use static methods for direct class access.

Single World

Function	Description	C Syntax	Python Syntax
Init	Initializes manager with program arguments	`const MPMDManager* Manager_Init(char** argv)`	`MPMDManager.init() -> None`
Finalize	Cleans up manager resources	`void Manager_Finalize(const MPMDManager*)`	`MPMDManager.finalize() -> None`
Local Name	Gets local program name	`const char* Manager_Local_name()`	`MPMDManager.local_name() -> str`
Local Comm	Gets local communicator	`const MPI_Comm* Manager_Local_comm()`	`MPMDManager.local_comm() -> MPI.Intracomm`
Local Rank	Gets process rank	`int Manager_Local_rank()`	`MPMDManager.local_rank() -> int`
Local Size	Gets program size	`unsigned int Manager_Local_size()`	`MPMDManager.local_size() -> int`
Comm To	Gets remote communicator	`const MPI_Comm* Manager_Comm_to(ConnectionId, IdType)`	`MPMDManager.comm_to(identifier: str \| int) -> MPI.Comm`
Size Of	Gets remote program size	`unsigned int Manager_Size_of(ConnectionId, IdType)`	`MPMDManager.size_of(identifier: str \| int) -> int`

Multi World (Client)

Function	Description	C Syntax	Python Syntax
Init	Initializes client manager	`const ClientManager* Client_Init(uint maxServers)`	`ClientManager.init() -> None`
Finalize	Cleans up client resources	`void Client_Finalize(const ClientManager*)`	`ClientManager.finalize() -> None`
Connect	Connects to named server with retry logic	`const MPI_Comm* Client_Connect(const char, const ConnectAttemptData)`	`ClientManager.connect(server_name: str, attemptData=ConnectAttemptData()) -> MPI.Intercomm`
Retrieve Server Comm	Gets server communicator	`const MPI_Comm* Client_Retrieve_Server_comm(ConnectionId, IdType)`	`ClientManager.retrieve_server_comm(server_id: str \| int) -> MPI.Intercomm`
Disconnect Server	Disconnects from specific server	`void Client_Disconnect_server(ConnectionId, IdType)`	`ClientManager.disconnect_server(server_id: str \| int) -> None`
Disconnect Servers	Disconnects from all servers	`void Client_Disconnect_servers()`	`ClientManager.disconnect_servers() -> None`

Multi World (Server)

Function	Description	C Syntax	Python Syntax
Init	Initializes server manager	`const ServerManager* Server_Init(char*, const char, uint)`	`ServerManager.init(server_name: str \| None = None) -> None`
Finalize	Cleans up server resources	`void Server_Finalize(const ServerManager*)`	`ServerManager.finalize() -> None`
Local Name	Gets server name	`const char* Server_Local_name()`	`ServerManager.local_name() -> str`
Open	Opens server for connections	`void Server_Open()`	`ServerManager.open() -> None`
Close	Closes server	`void Server_Close()`	`ServerManager.close() -> None`
Accept	Accepts client connection	`const MPI_Comm* Server_Accept(const char*)`	`ServerManager.accept(client_id: str \| None = None) -> Intercomm`
Retrieve Client Comm	Gets client communicator	`const MPI_Comm* Server_Retrieve_Client_comm(ConnectionId, IdType)`	`ServerManager.retrieve_client_comm(client_id: str \| int) -> MPI.Intercomm`
Disconnect Client	Disconnects specific client	`void Server_Disconnect_client(ConnectionId, IdType)`	`ServerManager.disconnect_client(client_id: str \| int) -> None`
Disconnect Clients	Disconnects all clients	`void Server_Disconnect_clients()`	`ServerManager.disconnect_clients() -> None`

📜 Certification

This software is officially registered with the Brazilian National Institute of Industrial Property (INPI) as a Computer Program:

Certificate Type: Certificate of Registration of a Computer Program
Registration Number: BR512025000477-0
Issuance Date: February 2025
Expiration Date: January 2076
Certificate URL: MPMDManager Certificate
Issuing Authority: INPI Brazil

📝 License

This project is MIT Licensed. See LICENSE file for more details.

👨‍💻 Authors

Ríad Oliveira Paulo Henrique

Ríad Oliveira de Morais

Full Stack Developer

MPMDManager

MPMDManager

Contents

💡 Proposal

🛠️ Technologies

🚀 Getting Started

Prerequisites

Installation

Project Structure

Running Examples

⚙️ Features

Single World

Multi World (Client)

Multi World (Server)

📚 API Reference

Notes

Single World

Multi World (Client)

Multi World (Server)

📜 Certification

📝 License

👨‍💻 Authors