Proceedings of the International Conference on Communication and Electronics Systems (ICCES 2018)
IEEE Xplore Part Number:CFP18AWO-ART; ISBN:978-1-5386-4765-3
Preprocessing Using Attribute Selection in
Data Stream Mining
R.Sangeetha
Research Scholar Dept. of Computer Science
Erode Arts and Science College
Erode, India
geethsanmca@gmail.com
Abstract - Data stream mining has enormous, increasing,
dynamic set of data with the time field registered while
entering the information in online. It includes numerous
attributes which ease the communication and ordering
process between the customer and commercial centre. The
basic principle in mining is to analyze the data in variant
perspectives. The need for exploring such data into useful
information in the concept of streaming in online and offline
grow the challenge into a major thing. Data collection and
preprocessing are the essential and earlier stage in mining.
The method used in preprocessing enhances and ease the
process unless it will lead to a difficult mining while gaining
knowledge about the data. Integrating and transforming the
data into an understandable format needs efficient
preprocessing tools and procedures. This paper explores the
preprocessing methods by applying Attribute Selection in
WEKA tool that aids in a simplified and structured set of
information. The proposed Attribute selection method
removes the irrelevant attribute by using Cfs Subset
evaluator with Greedy search method. Finally the size of the
file before and after preprocessing, number of attribute
elevated through this method is listed as the performance
evaluation.
Keywords – Data stream mining, Preprocessing, Attribute
selection, Correlation coefficient, Greedy search.
I. INTRODUCTION
Data Stream mining referred as analysing high speed
enormous data with no limit. [1] It has the feature of continual
arrival, multiplicity, rapid, time variant, boundless,
unpredictable streams that arise new research problems and
challenges. Analyzing this streaming model turn the researchers
into a significant attention in the past few years. The challenge
includes memory limitation, faster computing in just one pass
etc. So the extraction of the hidden knowledge requires
automated efficient techniques. However the classic data mining
procedures won’t fit for stream mining since it involves multiple
pass over the data to explore the patterns. The pattern also keeps
changing over the time so it should be recorded. As the volume
978-1-5386-4765-3/18/$31.00 ©2018 IEEE
Dr. S. Sathappan
Associate Professor, Dept. of Computer Science
Erode Arts and Science College
Erode, India
devisathappan@gmail.com
of data is bulky in the case of streaming, the resource allocation
is a big challenge. In order to minimize this issue data
preprocessing is required in which the irrelevant attributes,
duplicate instances are removed to reduce the file size and
memory space.
Preprocessing
Preprocessing is a procedure for Cleaning, Integrating,
Transforming the data before further processing. [2] Pre
processor is a program that processes the raw data to the clean
and fine set that is used as input to another program. The data
contains impurities, missing values, redundancies that is not
applicable for the direct processing as it leads to low accuracy,
high complexity. Streaming data has a list of attributes for the
customer convenience but all those are not needed for the
analysis. So preprocessing technique eliminates the extraneous
attributes and instances in parallel it retain the set with most
relevant information. The final stage of data stream mining is
acquiring the knowledge but it cannot be easily obtained due to
the huge data growth and neither can be easily understood unless
preprocessing method is employed.
Raw data
Preprocessing methods
Data cleaning, Integration,
Selection, Reduction and
Transformation.
Pre processed data for
analysis
Figure 1. Data Preprocessing
Figure 1. Shows the Data Preprocessing methods. It consists
of Cleaning, Integration, Selection and transformation.
Steps in Preprocessing:
A. Data Cleaning
The data may have incompleteness, noisy and not
consistent. There might be missing values and attributes. Also it
431
 Proceedings of the International Conference on Communication and Electronics Systems (ICCES 2018)
IEEE Xplore Part Number:CFP18AWO-ART; ISBN:978-1-5386-4765-3
contains errors and outliers. If the record is not pure, the results
would be neither reliable nor accurate.
This process involves techniques including:
Aggregation, Sampling, Remove, Fill the missed value, Fill the
missing attribute etc.
Strategies:
Missing Values
The file can have more missing entries and it make way to
misclassification.
Methods to handle missing values
1. Ignoring the tuple: This is taken when the label is
missing. While ignoring the tuple, the remaining
attributes for that particular is also not used. But this
not effective, if the tuple does not contains several
attributes with missed values.
2. Filling the missed value: The values are entered
manually and is time consuming.
3. Replace the value: Replace all values by a unique
constant. This method is easy and is perfect.
4. Use a mean or median to fill value: which pick the
“middle” value of a series to fill the value. The
symmetric distributions require the mean value, while
skewed distribution requires median to fill or replace.
5. Use the most possible value to fill the value: Using a
“Bayesian method or decision tree” induction to find
the most possible value.
Noisy Data
Noise is said as a “random error or variance” in a measured
variable.
Methods to handle noisy data:
1. Binning: These methods have a sorted data based on
the values around it. Then the values are allotted into a
number of “bins”. It is a variation that each value is
exchanged with the mean.
2. Regression: Data smoothing can also done by
regression. It assigns the values to a function. Linear
regression finds the “best” two attributes so that one
can be used to foresee the other. Multiple linear is a
variation in which multi attributes are selected and are
distributed in a multi-dimensional surface.
3. Outlier analysis: It is analysed by clustering. The
unique values are formed as a unit in which the most
unfit one forms an outlier. The output of the cleaning
process is a structured one.
B. Data Integration
The facts from different sources are integrated into one unit.
They are in different formats in different sites. The sources may
be ‘files, spread sheets, documents, data cubes, and internet’
then on. Integration is a vital task because they are from
different sources and does not match. It is really difficult to
confirm that whether the entities in two different sources have
978-1-5386-4765-3/18/$31.00 ©2018 IEEE
the same value or not. To reduce errors, integration process
‘Metadata’ is used. The most usual issue in the dataset is
redundancy. The same fact may be available in different dataset
or even in different sources. Without affecting the reliability, the
integration process tries to reduce redundancy.
Strategies:
1. Manual Integration – User operate with the relevant
information.
2. Application Integration – A particular application do all
the process.
3. Middleware Integration – The integration process is
transferred to the middleware.
4. Virtual Integration –Data resides in the system and
have a set of unified view for accessing.
5. Physical Integration – Creates an advanced system and
has a Xerox from the base system.
C. Data Selection and Reduction
The relevant data is retrieved for analysis. As the
information is collected from variant sources it must be selected
based on the requirements. Not all the field is needed for the
particular analysis. So the relevant record is to be selected with
its attributes before it is mined all the others gets reduced.
Strategies:
Data Selection -
Relevant attribute is selected while the others are eliminated by
manually or by procedures. It creates a subset of meaningful
attributes. Reduces the inputs for processing and analysis, or
finding the most meaningful inputs.
Selection methods:
1. Filter method: Features are selected on the basis of their
scores in various statistical tests for their correlation
with the outcome variable.
2. Wrapper method: A predictive model used to evaluate a
combination of features and assign a score based on
model accuracy.
3. Embedded method: Embedded methods assess the best
contributing feature for the accuracy of the model while
it is being created.
Data Reduction
Data reduction techniques are utilized to obtain a reduced
representation of the data set that is much smaller in dimension
but still contain important information.
Reduction methods:
1. Dimensionality reduction
It lessen the number of variables or attributes on some
consideration.
And it includes,
a. Wavelet transforms: It transforms a vector into a
numerically different vector with wavelet coefficients.
Wavelet transformed vector can be truncated.
432
 Proceedings of the International Conference on Communication and Electronics Systems (ICCES 2018)
IEEE Xplore Part Number:CFP18AWO-ART; ISBN:978-1-5386-4765-3
b. Principal Components Analytics: Selects the important
attributes.
c. Attribute subset formation: Redundant attributes are
removed by forming subset.
2. Numerosity reduction
This will replace the initial volume with the other forms of
data representation for its simplest form.
It may be,
a. Parametric: Data parameters are stored instead of actual
data.
b. Non-parametric: Stores simplified forms of data with
histograms, sampling.
3. Compression
The transformations are put to get a “reduced or
compressed” form of the prior.
It may be,
a. Lossless: Primary info can be recreated without loss.
b. Lossy: Cannot recreate the primary one, only an
approximation will be get.
There are numerous lossless algorithms for compression,
and they allow only limited data manipulation. Dimensionality
reduction and numerosity reduction is also considered one form
of compression.
D. Data Transformation
The record is in different formats as the sources are different.
It might be transferred into suitable one for mining. This process
includes ‘Generalization, Aggregation, Normalization etc. The
database can have mixed data types as integer and floating point
data. Before it is mined it might be change into either one form
for consistency.
Transformation Strategies
1. Smoothing: It removes noise. Techniques include
‘binning, regression, and clustering’.
2. Attribute construction: New attributes are created and
added
3. Aggregation: Constructs a data cube for analysis.
4. Normalization: Attributes are scaled with a mini range.
It assigns an equal numbers to all the attributes.
5. Discretization: Normal values are redirected by
intervals.
6. Hierarchy generation: Attributes are modified from
higher to lower level.
The remaining section of this paper is organized as:
Section II describes the Attribute/Feature selection
Procedures in preprocessing, Section III explains the related
works, Section IV elaborates the existing and the proposed
preprocessing procedure, Section V list the experimental results
with evaluation metrics and Section VI concludes the survey and
enlist the future work.
978-1-5386-4765-3/18/$31.00 ©2018 IEEE
II. ATTRIBUTE/FEATURE SELECTION IN
PREPROCESSING
Data streaming has tremendous growth in data as per the
customer and electronic commerce communication. The data
should be involved in classification, clustering techniques to get
the meaningful pattern. [3] The representation of data often
consists of too many features, but only a few of them might be
related to the concept.The feature that explains the data should
be limited by elevating the irrelevant, redundant to ease the
process in mining. For that easiness attribute selection
procedures in data preprocessing is required.
Features could be of any type such as discrete, continuous or
nominal. Selection is a process of selecting a subset of original
features according to certain criteria. It increases the speed and
accuracy of mining algorithm.
Objectives of attribute selection:
a. To simplify the models
b. To reduce training times
c. To lessen the difficulties of dimensionality
d. To enhance the process
e. To improve the accuracy
f. To avoid over fitting
Procedure Feature subset
Step 1: Start with all features.
Step 2: Apply evaluator for the generation of feature
subset.
Step 3: Put in search method for the evaluation of the
subset.
Step 4: Evaluate the Stopping Criteria.
Step 5: Validate the subset.
Description
1. Generation of the feature subset
It begins with the all the set of features with the search
strategy. The exhaustive search needs to search all of 2n possible
subsets of n features.
Characteristics of feature subset selection
The following characters should be identified and removed.
a. Relevant: These features have an influence on the
output and it has the significant role in describing the
data.
b. Irrelevant: These features do not have any effect on the
outcome has no vital role.
c. Redundant: A redundancy exists whenever a feature
takes the role of another feature.
433
 Proceedings of the International Conference on Communication and Electronics Systems (ICCES 2018)
IEEE Xplore Part Number:CFP18AWO-ART; ISBN:978-1-5386-4765-3
2. Subset Evaluation
After generating subsets of features, it should be evaluated.
There are three types of approaches namely Filter, Wrapper and
Embedded used as an evaluator.
3. Stopping Criteria
A stopping criterion has control on the flow of the procedure
and determines when the process must terminate. It is done
using a search algorithm. It is necessary to halt the search after
the number of selected features reached a pre-determined
threshold value. After an iterative process, the best subsets
among the generated candidates are selected.
Frequently used stopping criteria:
 Reach the end and the search gets completed.
 Reach the specified boundary which is set as minimum
number of features or maximum number of iterations.
 Find out reasonable good number of good subset.
4. Validation
The straight forward method for validating is to apply
learning algorithm by measuring the result with some prior
knowledge about the description of the features.
Advantages of Attribute Selection Procedure
 Improves the quality of the model.
 Reduction in memory space.
 Able to discover meaningful pattern in limited search.
 Reduction in high to low dimensionality leads to lessen
the complexity and improves the accuracy.
III. LITERATURE REVIEW
Poonam et al [4] did a survey on data stream mining.
Traditional dataset stores records without the concept of time
unless it is added explicitly. Data stream mining differs as it
includes real time data with the time concept. So for this online
analysis variant and efficient techniques should be employed.
This paper find a prominent streaming technique by having a
theoretical analysis on stream mining.
Divya Tomar et al [5] made a survey on Preprocessing and
Post processing technique in data mining. The data may include
several inconsistencies, missing values and irrelevant data.
These all removed with the use of preprocessing which is carried
earlier. This paper elaborates the pre and post processing with
various methods. Also it describes three visualization tools as it
is vital in exploring the data.
Francisca Rosario et al [6] put forth a preprocessing
procedure to select the significant attribute from the whole set.
Relief is a feature selection algorithm for random selection of
feature. This algorithm adopts the random selection for weight
estimation. The attribute weight estimated by Relief has a
probabilistic interpretation. A sample is selected, and the nearest
hit and nearest miss are identified. From the result it shows the
978-1-5386-4765-3/18/$31.00 ©2018 IEEE
accuracy seems to be better by applying the preprocessing Relief
procedure.
Uma K, M. Hanumanthappa et al [7] proposed data
collection and preprocessing methods in health care set. This set
contains structured and unstructured data and it combines text
and images in a single file. This type of heterogeneous data
needs a best preprocessing tool for the cleaning of data for
further analysis. This paper discusses the collection method and
types of preprocessing technique and its need. For handling the
missing value this paper employs the imputation method and
finally data reduction method is used to eliminate the complexity
and cost.
Tolga Dimergan et al [8] did a analyze on performance and
achievement analysis on student with preprocessing technique in
WEKA. The information was created through the voluntary
participation of Trakya University. 51 questions were framed
against 156 students to assess. The contribution of each attribute
is found by the Select Attributes algorithms provided by WEKA.
In this study, Info-Gain Attribute Evaluation was used to reveal
how much information is provided by each attribute of each
instance in the set. Finally five most effective attribute were
selected through this method.
IV. METHODOLOGY
Existing Preprocessing Procedure:
The existing procedures include Information gain and Relief-
F selection measure as preprocessing methods for feature or
attribute selection.
A. Information Gain (IG) – Attribute selection
To determine the best attribute the measure called
Information Gain is used. [8] A common measure for the
information is Shannon entropy.
Entropy measure:
Information entropy is the average rate at which
information is produced by a source of data. The measure of
information entropy associated with each possible data value is
the negative log value of the probability mass function given in
(1). Thus, the data with less probability carries more information
than the higher-probability value.
The Entropy of Y is,
--- (1)
Where pi is the probability of the random variable Y.
If the observed values in the training data set D are partitioned
according to the values of a second attribute X, and the entropy
of Y with respect to the partitions induced by X is less than the
entropy of Y prior to partitioning, then there is a relationship
between the attributes Y and X as given in (2).
The Entropy of Y after observing X is,
--- (2)
434
 Proceedings of the International Conference on Communication and Electronics Systems (ICCES 2018)
IEEE Xplore Part Number:CFP18AWO-ART; ISBN:978-1-5386-4765-3

Then Information Gain (IG) is calculated as,
--- (3)
The equation in (3) shows that information gained about Y after
observing X is equal to the information gained about X after
observing Y.
Step 2: Randomly select the instances of features.
Step 3: Find the nearest hit and nearest miss for
the randomly selected instances features
based on Euclidean distance.
Step 4: Calculate the weight for the feature.
Step 5: Select the feature above the threshold value.
Step 6: Add the selected to the feature set.

Procedure IG Advantages:
Step 1: Initialize the feature set with zero.  More robust with noisy data.
Step 2: Calculate Shannon Entropy for the class.  Handle multi-class problems,
Step 3: Calculate Entropy for the attribute values.  Robust with incomplete data.
Step 4: Measure the conditional probability of each Disadvantages:
term.  The estimation of features using Euclidean distance
Step 5: Select the term with highest information gain. which uses mean value will have some negative effect
Step 6: Add the term to the feature set. if the instances of features are outliers.

Advantage: Proposed Preprocessing Procedure:

 Significantly improves the accuracy and complexity by The proposed method employs Attribute selection method
eliminating irrelevant attributes. which involves Correlation feature subset (Cfs) evaluator with
Disadvantage: Greedy forward search method. The Cfs is based on filter based
 It is biased in favour of attributes with more values method.
even when they are not more informative. Filter Method:
Filter model relied on analyzing the general characteristics of
B. Relief-F– Attribute Selection data [3] without involving any learning algorithms and is free
It is expanded as “Relief Feature”.Relief-F is a feature from bias associated with any learning models.
selection algorithm for random selection of feature with weight
calculation. This algorithm weights each feature according to its Dataset with Feature subset
relevance to the class. [6] Initially, all weights are set to zero and all features evaluator + Subset of
then updated iteratively. A sample is selected from the data, and Search features
the nearest neighbor sample that belongs to the same class method
known as nearest hit and the nearest neighboring sample that
belongs to the opponent class known as nearest miss are Learning
algorithm
identified. A change in attribute value accompanied by a change
in class leads to increase in weighting of the attribute and by no
change in class leads to decrease in weighting of the attribute. Knowledge gaining
This procedure of updating the weight of the attribute is
performed for a random set for every sample in the data. The
weight updates are then averaged so that the final weight is in Figure 2. Filter method
the range between −1, 1. The attribute weight estimated in (4) Figure 2. Shows the steps in evaluating filter method. This
has a probabilistic interpretation. It is proportional to the approach is efficient and fast to compute.
difference between two conditional probabilities, namely,
nearest miss and nearest hit respectively. Proposed Filter based Evaluator:
The weight is updated by, In the proposed method the features are estimated by the
score and the feature with the best scores are utilized in building
the model, while others are not used for analysis. Usually the
--- (4) features are ranked with score by Pearson correlation as in (5). It
Where, Wi = Weight vector; xi = Feature vector. is computed by calculating the covariance of two variables and
Nearhit and nearmiss are calculated using divide by the product of their standard deviations.
Euclidean distance.
Cfs subset Evaluator:
Procedure Relief-F It is expanded as “Correlation feature subset.”
Step 1: Initialize the weight with zero.

978-1-5386-4765-3/18/$31.00 ©2018 IEEE 435

 Proceedings of the International Conference on Communication and Electronics Systems (ICCES 2018)
IEEE Xplore Part Number:CFP18AWO-ART; ISBN:978-1-5386-4765-3
It evaluate the [9] worth of a subset by considering the unique
predictive ability of each attribute along with the degree of
redundancy. It starts with discretization which means the
process of converting large number of data values into a smaller
one. The rest of features should be ignored. Correlation is a
well-known similarity measures to gain information between
two features. If two features are linearly dependent, their
correlation coefficient is ±1. If the they are uncorrelated, the
correlation coefficient is 0.
Hypothesis:
“A good feature subset is one that contains features which are
highly correlated within the class, but uncorrelated with the
rest.”
This aid in two definitions.
Feature-class correlation – Indicates how much a feature is
correlated to a specific class.
Feature-Feature correlation – Indicates the correlation between
two features.
Pearson Correlation:
--- (5)
Equation (5), states the merit of a feature subset.
Where,
k - Number of features in the set.
Merits - Having the group of attribute S including k
attribute that chosen from the group.
Rcf - is the average value of the groups of chosen
attributes from all attribute which have related
relationships with the type of data.
Rff - is the average value of groups of chosen
attributes from all attributes which have the
inter-related relationship in the same group of
chosen attributes.
Here, rcf is average feature-class correlation, and rff is average
feature-feature correlation.
Proposed search method:
A search method is a stage-by-stage procedure used to gain
significant data among a collection. It is considered a
fundamental procedure in computing. The efficiency is
measured by the number of times a comparison of the search is
done in the worst case. The search algorithm depends on the data
structure, and prior knowledge about the data.
Greedy forward stepwise search method:
Greedy Approach is based [10] on the concept of heuristic
Problem by making optimal local choice at each node. By
making these local optimal choices, the optimal solution is
reached.
The algorithm can be summarized as:
978-1-5386-4765-3/18/$31.00 ©2018 IEEE
1. At each stage, pick out the best attribute usually the nominal
class value as the test condition.
2. Now split the node into the possible outcomes
3. Repeat the above steps till all the test conditions have been
exhausted into leaf s.
It is the simplest greedy algorithm. A greedy algorithm is a
paradigm of making the locally optimal choice at each stage.
There is no backtracking. Greedy may start when there are no or
all attribute subsets present or it in between the attribute subset
searching for the best feature. By traversing the space it produce
a ranked list of attributes and records the attribute which are
selected. It starts from the empty set and the features which
satisfies the objective function and update the final set.
In general, greedy algorithms have five components:
i. A candidate set, from which a solution is created.
ii. A selection function, which chooses the best candidate
to be added to the solution.
iii. A feasibility function that is used to determine if a
candidate can be used to contribute to a solution.
iv. An objective function, which assigns a value to a
solution, or a partial solution.
v. A solution function, which will indicate when we have
discovered a complete solution.
Procedure Cfs with Greedy
Step 1: Initialize the feature subset with empty.
Step 2: Discretization with the training dataset.
Step 3: Calculate Pearson correlation with the set.
Step 4: Find feature-class and feature- feature
correlation.
Step 5: Select the feature with highly correlated.
Step 6: Add it to the feature subset.
Step 7: Initialize the candidate set with x entries
obtained by the correlation feature subset.
Step 8: Initialize optimum solution set to empty.
Step 9: Select entry from the candidate set using
Greedy strategy.
Step 10: If it is optimum with high values, move to
optimum solution set, else remove it.
Step 11: Repeat step 9 until the candidate set is
empty.
Advantages of Cfs with Greedy procedure
a. Fast and easy to make with filter based.
b. Greedy takes decision in every step.
c. Never backtracks.
d. Lessen the storage space.
e. Minimize the time and complexity in further mining
process by reducing the feature set.
436
 Proceedings of the International Conference on Communication and Electronics Systems (ICCES 2018)
IEEE Xplore Part Number:CFP18AWO-ART; ISBN:978-1-5386-4765-3

Disadvantage of Cfs with Greedy procedure

a. For large and multidimensional it won’t give optimum
solution. Data set after Preprocessing
b. It takes longer time for big data.

V. EXPERIMENTAL RESULTS

This work is carried out in WEKA 3.8.6. “Waikato [11]

Environment for Knowledge Analysis” is a machine
learning software package coded in Java, built at the ‘University
of Waikato, New Zealand’. It supports solely ‘ARFF’ files. The
excel file are often simply transform to ARFF format via ‘CSV’
format. Also WEKA provides access to SQL and the result came Figure 4. Attributes after Preprocessing
back by a database query. Figure 4. Shows the attribute list screen after preprocessing in
WEKA. There are five attributes.
Data set
The Online Retail dataset is taken from the UCI [12] Evaluation
repository. The dataset has the information about one week Table 2. Evaluation metrics
online ordering details with eight attributes and 9974 instances Metrics Existing Existing Proposed
in UK and is a registered online retail store. The store uniquely Procedure 1 Procedure 2 Procedure
sells all occasional gifts. Many customers are wholesalers. (ReliefF) (Information (Correlation
Gain) feature
Data set description subset)
Table 1. Dataset description Number of 6 – A1, 6 – A1, 5 – A2,
Attribute Attribute Attribute attribute A2, A3, A4,
Number Name Description selected A3, A4, A6,
A1 Ivno Invoice Number A5, A5, A7,
A2 Scode Stock code A7, A7, A8.
A3 Product Name of the A8. A8.
Product Selected Ivno, scode, Ivno, scode,
A4 Quantity Ordered quantity Attribute Product, Product, quantity,
A5 Ivdte Invoice date cusid, Ivdte, quantity, uprice,
A6 Uprice Unit price country. Ivdte,cusid, cusid,
A7 Cusid Customer identity country. country.
A8 Country Customer country Handling Binary, date, Binary, Binary, date,
Class Numeric, Nominal, Numeric,
Table 1. list out the number of attributes, with its name and value type Nominal, Missing Nominal,
description. Misingvalue. value. Misingvalue.
File size in 822 / 751 822 / 716 822 / 228
Data set Before Preprocessing Kb Before
/ After
Preprocess
ing

Table 2. list out the performance of the existing and

proposed procedures with the evaluation metrics. Attributes A1,
A2,..A8 are described in Table 1.

Figure 3. Attributes before Preprocessing

Evaluation Metrics Description:
Number of attribute selected
Figure 3. Shows the attribute list screen before In the existing procedures 1 and 2 six are selected while in
preprocessing in WEKA. There are eight attributes. the proposed five relevant are selected.

978-1-5386-4765-3/18/$31.00 ©2018 IEEE 437

 Proceedings of the International Conference on Communication and Electronics Systems (ICCES 2018)
IEEE Xplore Part Number:CFP18AWO-ART; ISBN:978-1-5386-4765-3
Selected Attribute
In the proposed procedure the selected attribute solely define
the data set in a meaningful way. There is no redundant
description.
In Existing procedure 1 selected attribute Scode and Product
have the redundant information. Both describes the product,
In Existing procedure 2 selected attribute Ivno and Ivdte have
redundant information.
In Proposed procedure the relevant attribute with no
redundancy is selected and the five attribute gives in depth
knowledge about the dataset.
Capabilities in handling class value type
Existing procedure 1 can handle variant class types.
Existing procedure 2 can handle three types of class values.
Proposed procedure can handle variant class types.
File Size Before/ After Preprocessing
The file size before preprocessing is 822 kb. It is reduced
while eliminating the attributes in the preprocessing. It will ease
the further performance and may need limited amount of storage
space.
In Existing set the Product and IVdte is selected leads to high
storage space.
The Proposed procedure lessens the size of the file to a
maximum extent by elevating the insignificant attribute and the
attribute which gives redundant information, in parallel it retains
the most relevant attribute. While elevating, the proposed
procedure removes the Product name and invoice date which has
high bits storage than the rest attribute and so the size is
minimized to the maximum extent. The name of the product can
be obtained by the code (Scode) of the product.
VI. CONCLUSION AND FUTURE WORK
Data stream contains dynamic, fast growing and
unstructured data in online. Mining these kinds of data requires
highly efficient technique. Preprocessing eases this mining by
removing irrelevant information and reduces the size and
complexity involved. Attribute selection procedure which has
Cfs subset evaluator with Greedy search method is employed as
a preprocessing procedure. The evaluation metrics are measured
with the existing and proposed procedures. The proposed
procedure in WEKA performs better than the existing by
retaining the most relevant by reducing the number of attribute
and file size.
This preprocessing work will be carried in future for further
data mining or stream mining tasks such as Classification,
Clustering for analysing the underlying hidden pattern.
978-1-5386-4765-3/18/$31.00 ©2018 IEEE
REFERENCES
[1] Agarwal C., “Data Streams model and algorithms”, Springer science
and Business Media, Volume 31, 2007.
[2] J.Gama, “Knowledge discovery from data streams”, Chapman and
Hall /(RC, 2010.
[3] K,Sudha, Dr.J. Jebamalar Tamilselvi, “A Review of Feature Selection
Algorithms for Data Mining Techniques”, International Journal on
Computer Science and Engineering (IJCSE), Vol. 7 No.6 Jun 2015.
[4] Poonam Debnath, Santosh kumar Chobe, “A Quick Survey on Data
Stream Mining”, International Journal of Computer Science and
Information Technologies, Vol. 5 (3), 2014.
[5] Divya Tomar, Sonali Agarwal, “ A survey on Preprocessing and Post
Processing technique in data mining”, Intrnational Journal of
Database theory and applications, Vol 7, No.4, 2014.
[6] S. Francisca rosario, Dr. K. Thangadurai, “Relief: feature selection
approach “International journal of innovative research &
development, vol 4, issue 11, 2015.
[7] Uma K, M. Hanumanthappa, “Data Collection Methods and Data
Preprocessing Techniques for Healthcare Data Using Data Mining”,
International Journal of Scientific & Engineering Research Volume 8,
Issue 6, June-2017.
[8] Tolga Demirhan, Ilker Hacioglu, “Performance and achievement
analysis of a dataset of distance education samples with weka”, The
Eurasia Proceedings of Educational & Social Sciences (EPESS),
Volume 8, 2017.
[9] M.A. Hall, “Correlation-based Feature Selection for Machine
Learning “, department of computer science, The University of
Wikato, Hamilton Newzealand, 1999.
[10] http:// en.wikipedia.org/wiki/Greedy algorithm.
[11] Eshwari Girish Kulkarni, Raj B. Kulkarni, PhD, “WEKA Powerful
Tool in Data Mining”, International Journal of Computer
Applications (0975 – 8887) National Seminar on Recent Trends in
Data Mining RTDM 2016.
[12] C. Blake, E. Keogh, C.J. Merz, “UCI repository of Machine Learning
Database”.
438