How can PHP be used to find duplicates in a table based on percentage similarity?

To find duplicates in a table based on percentage similarity in PHP, we can use the Levenshtein distance algorithm to calculate the similarity between two strings. We can compare each pair of strings in the table and if the similarity percentage exceeds a certain threshold, we can consider them as duplicates. 

<?php
// Function to calculate similarity percentage between two strings
function similarityPercentage($str1, $str2) {
    similar_text($str1, $str2, $similarity);
    return $similarity;
}

// Connect to the database
$servername = "localhost";
$username = "username";
$password = "password";
$dbname = "database";
$conn = new mysqli($servername, $username, $password, $dbname);

// Select data from the table
$sql = "SELECT id, column_name FROM table_name";
$result = $conn->query($sql);

// Loop through the results and find duplicates based on similarity percentage
while($row = $result->fetch_assoc()) {
    $id = $row["id"];
    $string1 = $row["column_name"];
    
    $result2 = $conn->query($sql);
    while($row2 = $result2->fetch_assoc()) {
        $id2 = $row2["id"];
        $string2 = $row2["column_name"];
        
        if($id != $id2) {
            $similarity = similarityPercentage($string1, $string2);
            if($similarity > 80) { // Adjust the threshold percentage as needed
                echo "Duplicates found: $id and $id2\n";
            }
        }
    }
}

$conn->close();
?>

Keywords

PHP similarity duplicates table percentage

Related Questions