Please do text matching according to this project
My project only provides a simple AI text matching module, which can match the text of the same meaning with special or invalid characters
E.g
similarity(‘gender’, ‘gen@’)=0.6
similarity(‘gender’, ‘gendder’)=0.923
similarity(‘gender’, ‘_gendder’)=0.857
Treat other completely wrong text as invalid value
For a stronger AI text matching module, such as supporting text matching in different languages, please copy the https://github.com/UKPLab/sentence-transformers project
The https://github.com/UKPLab/sentence-transformers project is too complicated. If you want to quickly implement multi-language matching, you can use python to call Google Translate for automatic translation.
First install googletrans:
pip install googletrans
Then, call the following code to translate the header of any language into English.
from googletrans import Translator
string = the header of any language
translater = Translator()
out = translater.translate(string, dest=\’en\’, src=\’auto\’).text
Then use the similarity(\’gender\’, \’gendder\’)=0.923 function to perform text matching.
Use regular expressions in the python language to automatically identify commas, spaces, semicolons, vertical bars and other delimiters, and save them as comma delimiters csv
import pandas as pd
csv = pd.read_csv(\'input.csv\', sep=\'\\s|,|;\')
csv.to_csv(\'result.csv\')
