DATA PROCESSING
DATA PROCESSING PROCESS
There are four basic data processing techniques to improve the performance of AI models : 1 | Data selection 2 | Data structuring 3 | Data quality evaluation 4 | Optimisation
This can be an easy task to complete , if you want to extract data from an excel sheet , for instance . With numbers neatly put into their respective and labelled columns and rows , getting a machine to extract , interpret and take action on is relatively easy .
“ The problem on the other hand , let ’ s say we want to use AI to provide a score for how positive or negative each comment on X ( formerly Twitter ) was about a specific company ,” says Alan Jacobson , Chief Data and Analytics Officer , Alteryx . “ We likely would not have shared agreements as humans assessing the words , let alone agreeing with a computer method of scoring . This fuzziness is where the challenges arise .” This is the issue with dealing with unstructured and heterogeneous data sources in AI projects . There are insights to be gained , but by putting them into a system unrefined can skew models .
To tackle this , companies can employ data cleaning methods . These tackle errors and inconsistencies , ensuring the information is accurate and complete before it goes to the system .
130 July 2024