What are some best practices for handling large amounts of data in PHP, specifically when sorting and moving data between databases?

When handling large amounts of data in PHP, especially when sorting and moving data between databases, it is important to optimize your code for performance. One way to do this is by using batch processing techniques to limit the amount of data being processed at once, reducing memory usage and improving efficiency. 

// Example of batch processing data from one database to another
$sourceDb = new PDO('mysql:host=localhost;dbname=source_db', 'username', 'password');
$targetDb = new PDO('mysql:host=localhost;dbname=target_db', 'username', 'password');

$batchSize = 1000;
$offset = 0;

do {
    $stmt = $sourceDb->prepare("SELECT * FROM table LIMIT $offset, $batchSize");
    $stmt->execute();
    $data = $stmt->fetchAll(PDO::FETCH_ASSOC);

    foreach ($data as $row) {
        // Process and insert data into target database
        $targetStmt = $targetDb->prepare("INSERT INTO table (column1, column2) VALUES (:value1, :value2)");
        $targetStmt->execute([':value1' => $row['column1'], ':value2' => $row['column2']]);
    }

    $offset += $batchSize;
} while (count($data) > 0);

Keywords

PHP large data sorting database migration optimization

Related Questions