iummili.blogg.se

We shall stop at a point where the data seems to have a reasonable number of similar observations based on the initial characters.

The iterative process will continue in the same fashion, and each time we need to pay more attention to the merged data to identify any incorrect merges as explained in step 2.Ĥ. Again, we shall retain the successfully merged records, append it to the already saved successful merged data, and delete it from the initial dataset, just we did in the first step above.ģ. So using the first 29 characters of the two variables, we shall proceed to merge. “THE BANK OF “, therefore, if we merge the two data sets using only the first 12 characters, we shall incorrectly merge THE BANK OF KHYBER into THE BANK OF PUNJAB THE BANK OF KHYBER For example, consider the following two records where the first 12 characters are exactly the same in both the records i.e.

But this time, we need to be careful as we reduce the extraction of the initial number of characters, the chances of matching incorrect records increases. The idea is that if two variables did not have the first 30 characters in common, they might have the first 29 characters in common. This time we shall extract one character less than what we used in the preceeding step.

In the next iteration, we shall further process those records which did not merge. We shall discuss it further as we proceed in this article.Ģ. Please note that the relevant Stata function is substr() for extracting a given number of characters from a variable. In this first step, we shall normally start with extracting a large number of characters, for example, up to 30 characters in the case of row 2 of the above table. Also, we shall delete, from the initial file, those records which were successfully merged, and further process those that did not merge. If the merge succeeds, we shall save the merged data separately. we start with extracting the first n-number of characters from both the key variables in the two datasets and merge using the extracted (truncated) variables. Instead, we shall take the iterative path where:ġ. So how exactly are we going to find the matching number of characters in each case? We shall not do that. If we count these characters, they are 30. For example, in the second row of the above table, “The Pakistan General Insurance” part is similar in both the tables. Given that, we can split the problem into two parts.Įxtract the first few characters that are similar in both the dataset and merge the data using those similar characters. However, there is a general patterns of similarity in the first few characters, starting from left to right. The above table shows that the company names not only differ in terms of different number of characters but also in terms of capitalization.