CVC Viagens

Postagens

asdoiasn

#importing basic packages import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt /usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead. import pandas.util.testing as tm #Loading the data data0 = pd.read_csv('5.urldata.csv') data0.head() Domain Have_IP Have_At URL_Length URL_Depth Redirection https_Domain TinyURL Prefix/Suffix DNS_Record Web_Traffic Domain_Age Domain_End iFrame Mouse_Over Right_Click Web_Forwards Label 0 graphicriver.net 0 0 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 1 ecnavi.jp 0 0 1 1 1 0 0 0 0 1 1 1 0 0 1 0 0 2 hubpages.com 0 0 1 1 0 0 0 0 0 1 0 1 0 0 1 0 0 3 extratorrent.cc 0 0 1 3 0 0 0 0 0 1 0 1 0 0 1 0 0 4 icicibank.com 0 0 1 3 0 0 0 0 0 1 0 1 0 0 1 0 0 3. Familiarizing with Data In this step, few dataframe methods are used to look into the data and its features. #Checking the shape of the dataset data0.shape (10000, 18) #Listing the features of the dataset data0.columns Index(['Domain', 'Have_IP', 'Have_At', 'URL_Length', 'URL_Depth', 'Redirection', 'https_Domain', 'TinyURL', 'Prefix/Suffix', 'DNS_Record', 'Web_Traffic', 'Domain_Age', 'Domain_End', 'iFrame', 'Mouse_Over', 'Right_Click', 'Web_Forwards', 'Label'], dtype='object') #Information about the dataset data0.info() RangeIndex: 10000 entries, 0 to 9999 Data columns (total 18 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Domain 10000 non-null object 1 Have_IP 10000 non-null int64 2 Have_At 10000 non-null int64 3 URL_Length 10000 non-null int64 4 URL_Depth 10000 non-null int64 5 Redirection 10000 non-null int64 6 https_Domain 10000 non-null int64 7 TinyURL 10000 non-null int64 8 Prefix/Suffix 10000 non-null int64 9 DNS_Record 10000 non-null int64 10 Web_Traffic 10000 non-null int64 11 Domain_Age 10000 non-null int64 12 Domain_End 10000 non-null int64 13 iFrame 10000 non-null int64 14 Mouse_Over 10000 non-null int64 15 Right_Click 10000 non-null int64 16 Web_Forwards 10000 non-null int64 17 Label 10000 non-null int64 dtypes: int64(17), object(1) memory usage: 1.4+ MB 4. Visualizing the data Few plots and graphs are displayed to find how the data is distributed and the how features are related to each other. #Plotting the data distribution data0.hist(bins = 50,figsize = (15,15)) plt.show() #Correlation heatmap plt.figure(figsize=(15,13)) sns.heatmap(data0.corr()) plt.show() 5. Data Preprocessing & EDA Here, we clean the data by applying data preprocesssing techniques and transform the data to use it in the models. data0.describe() Have_IP Have_At URL_Length URL_Depth Redirection https_Domain TinyURL Prefix/Suffix DNS_Record Web_Traffic Domain_Age Domain_End iFrame Mouse_Over Right_Click Web_Forwards Label count 10000.000000 10000.000000 10000.000000 10000.000000 10000.000000 10000.000000 10000.000000 10000.000000 10000.000000 10000.000000 10000.000000 10000.0000 10000.000000 10000.00000 10000.00000 10000.000000 10000.000000 mean 0.005500 0.022600 0.773400 3.072000 0.013500 0.000200 0.090300 0.093200 0.100800 0.845700 0.413700 0.8099 0.090900 0.06660 0.99930 0.105300 0.500000 std 0.073961 0.148632 0.418653 2.128631 0.115408 0.014141 0.286625 0.290727 0.301079 0.361254 0.492521 0.3924 0.287481 0.24934 0.02645 0.306955 0.500025 min 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.0000 0.000000 0.00000 0.00000 0.000000 0.000000 25% 0.000000 0.000000 1.000000 2.000000 0.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 1.0000 0.000000 0.00000 1.00000 0.000000 0.000000 50% 0.000000 0.000000 1.000000 3.000000 0.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 1.0000 0.000000 0.00000 1.00000 0.000000 0.500000 75% 0.000000 0.000000 1.000000 4.000000 0.000000 0.000000 0.000000 0.000000 0.000000 1.000000 1.000000 1.0000 0.000000 0.00000 1.00000 0.000000 1.000000 max 1.000000 1.000000 1.000000 20.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000 1.000000 1.00000 1.00000 1.000000 1.000000 The above obtained result shows that the most of the data is made of 0's & 1's except 'Domain' & 'URL_Depth' columns. The Domain column doesnt have any significance to the machine learning model training. So dropping the 'Domain' column from the dataset. #Dropping the Domain column data = data0.drop(['Domain'], axis = 1).copy() This leaves us with 16 features & a target column. The 'URL_Depth' maximum value is 20. According to my understanding, there is no necessity to change this column. #checking the data for null or missing values data.isnull().sum() Have_IP 0 Have_At 0 URL_Length 0 URL_Depth 0 Redirection 0 https_Domain 0 TinyURL 0 Prefix/Suffix 0 DNS_Record 0 Web_Traffic 0 Domain_Age 0 Domain_End 0 iFrame 0 Mouse_Over 0 Right_Click 0 Web_Forwards 0 Label 0 dtype: int64 In the feature extraction file, the extracted features of legitmate & phishing url datasets are just concatenated without any shuffling. This resulted in top 5000 rows of legitimate url data & bottom 5000 of phishing url data. To even out the distribution while splitting the data into training & testing sets, we need to shuffle it. This even evades the case of overfitting while model training. # shuffling the rows in the dataset so that when splitting the train and test set are equally distributed data = data.sample(frac=1).reset_index(drop=True) data.head() Have_IP Have_At URL_Length URL_Depth Redirection https_Domain TinyURL Prefix/Suffix DNS_Record Web_Traffic Domain_Age Domain_End iFrame Mouse_Over Right_Click Web_Forwards Label 0 0 0 1 2 0 0 0 1 0 1 0 1 0 0 1 0 1 1 0 0 1 5 0 0 0 0 0 1 0 1 0 0 1 1 0 2 0 0 1 1 0 0 0 0 0 1 1 0 0 0 1 0 0 3 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 0 0 4 0 0 0 0 0 0 0 1 0 1 0 1 0 0 1 0 1 From the above execution, it is clear that the data doesnot have any missing values. By this, the data is throughly preprocessed & is ready for training. 6. Splitting the Data # Sepratating & assigning features and target columns to X & y y = data['Label'] X = data.drop('Label',axis=1) X.shape, y.shape ((10000, 16), (10000,)) # Splitting the dataset into train and test sets: 80-20 split from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 12) X_train.shape, X_test.shape ((8000, 16), (2000, 16)) 7. Machine Learning Models & Training From the dataset above, it is clear that this is a supervised machine learning task. There are two major types of supervised machine learning problems, called classification and regression. This data set comes under classification problem, as the input URL is classified as phishing (1) or legitimate (0). The supervised machine learning models (classification) considered to train the dataset in this notebook are: Decision Tree Random Forest Multilayer Perceptrons XGBoost Autoencoder Neural Network Support Vector Machines #importing packages from sklearn.metrics import accuracy_score # Creating holders to store the model performance results ML_Model = [] acc_train = [] acc_test = [] #function to call for storing the results def storeResults(model, a,b): ML_Model.append(model) acc_train.append(round(a, 3)) acc_test.append(round(b, 3)) 7.1. Decision Tree Classifier Decision trees are widely used models for classification and regression tasks. Essentially, they learn a hierarchy of if/else questions, leading to a decision. Learning a decision tree means learning the sequence of if/else questions that gets us to the true answer most quickly. In the machine learning setting, these questions are called tests (not to be confused with the test set, which is the data we use to test to see how generalizable our model is). To build a tree, the algorithm searches over all possible tests and finds the one that is most informative about the target variable. # Decision Tree model from sklearn.tree import DecisionTreeClassifier # instantiate the model tree = DecisionTreeClassifier(max_depth = 5) # fit the model tree.fit(X_train, y_train) DecisionTreeClassifier(ccp_alpha=0.0, class_weight=None, criterion='gini', max_depth=5, max_features=None, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, presort='deprecated', random_state=None, splitter='best') #predicting the target value from the model for the samples y_test_tree = tree.predict(X_test) y_train_tree = tree.predict(X_train) Performance Evaluation: #computing the accuracy of the model performance acc_train_tree = accuracy_score(y_train,y_train_tree) acc_test_tree = accuracy_score(y_test,y_test_tree) print("Decision Tree: Accuracy on training Data: {:.3f}".format(acc_train_tree)) print("Decision Tree: Accuracy on test Data: {:.3f}".format(acc_test_tree)) Decision Tree: Accuracy on training Data: 0.810 Decision Tree: Accuracy on test Data: 0.826 #checking the feature improtance in the model plt.figure(figsize=(9,7)) n_features = X_train.shape[1] plt.barh(range(n_features), tree.feature_importances_, align='center') plt.yticks(np.arange(n_features), X_train.columns) plt.xlabel("Feature importance") plt.ylabel("Feature") plt.show() Storing the results: #storing the results. The below mentioned order of parameter passing is important. #Caution: Execute only once to avoid duplications. storeResults('Decision Tree', acc_train_tree, acc_test_tree) 7.2. Random Forest Classifier Random forests for regression and classification are currently among the most widely used machine learning methods.A random forest is essentially a collection of decision trees, where each tree is slightly different from the others. The idea behind random forests is that each tree might do a relatively good job of predicting, but will likely overfit on part of the data. If we build many trees, all of which work well and overfit in different ways, we can reduce the amount of overfitting by averaging their results. To build a random forest model, you need to decide on the number of trees to build (the n_estimators parameter of RandomForestRegressor or RandomForestClassifier). They are very powerful, often work well without heavy tuning of the parameters, and don’t require scaling of the data. # Random Forest model from sklearn.ensemble import RandomForestClassifier # instantiate the model forest = RandomForestClassifier(max_depth=5) # fit the model forest.fit(X_train, y_train) RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None, criterion='gini', max_depth=5, max_features='auto', max_leaf_nodes=None, max_samples=None, min_impurity_decrease=0.0, min_impurity_split=None, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, n_estimators=100, n_jobs=None, oob_score=False, random_state=None, verbose=0, warm_start=False) #predicting the target value from the model for the samples y_test_forest = forest.predict(X_test) y_train_forest = forest.predict(X_train) Performance Evaluation: #computing the accuracy of the model performance acc_train_forest = accuracy_score(y_train,y_train_forest) acc_test_forest = accuracy_score(y_test,y_test_forest) print("Random forest: Accuracy on training Data: {:.3f}".format(acc_train_forest)) print("Random forest: Accuracy on test Data: {:.3f}".format(acc_test_forest)) Random forest: Accuracy on training Data: 0.814 Random forest: Accuracy on test Data: 0.834 #checking the feature improtance in the model plt.figure(figsize=(9,7)) n_features = X_train.shape[1] plt.barh(range(n_features), forest.feature_importances_, align='center') plt.yticks(np.arange(n_features), X_train.columns) plt.xlabel("Feature importance") plt.ylabel("Feature") plt.show() Storing the results: #storing the results. The below mentioned order of parameter passing is important. #Caution: Execute only once to avoid duplications. storeResults('Random Forest', acc_train_forest, acc_test_forest) 7.3. Multilayer Perceptrons (MLPs): Deep Learning Multilayer perceptrons (MLPs) are also known as (vanilla) feed-forward neural networks, or sometimes just neural networks. Multilayer perceptrons can be applied for both classification and regression problems. MLPs can be viewed as generalizations of linear models that perform multiple stages of processing to come to a decision. # Multilayer Perceptrons model from sklearn.neural_network import MLPClassifier # instantiate the model mlp = MLPClassifier(alpha=0.001, hidden_layer_sizes=([100,100,100])) # fit the model mlp.fit(X_train, y_train) MLPClassifier(activation='relu', alpha=0.001, batch_size='auto', beta_1=0.9, beta_2=0.999, early_stopping=False, epsilon=1e-08, hidden_layer_sizes=[100, 100, 100], learning_rate='constant', learning_rate_init=0.001, max_fun=15000, max_iter=200, momentum=0.9, n_iter_no_change=10, nesterovs_momentum=True, power_t=0.5, random_state=None, shuffle=True, solver='adam', tol=0.0001, validation_fraction=0.1, verbose=False, warm_start=False) #predicting the target value from the model for the samples y_test_mlp = mlp.predict(X_test) y_train_mlp = mlp.predict(X_train) Performance Evaluation: #computing the accuracy of the model performance acc_train_mlp = accuracy_score(y_train,y_train_mlp) acc_test_mlp = accuracy_score(y_test,y_test_mlp) print("Multilayer Perceptrons: Accuracy on training Data: {:.3f}".format(acc_train_mlp)) print("Multilayer Perceptrons: Accuracy on test Data: {:.3f}".format(acc_test_mlp)) Multilayer Perceptrons: Accuracy on training Data: 0.859 Multilayer Perceptrons: Accuracy on test Data: 0.863 Storing the results: #storing the results. The below mentioned order of parameter passing is important. #Caution: Execute only once to avoid duplications. storeResults('Multilayer Perceptrons', acc_train_mlp, acc_test_mlp) 7.4. XGBoost Classifier XGBoost is one of the most popular machine learning algorithms these days. XGBoost stands for eXtreme Gradient Boosting. Regardless of the type of prediction task at hand; regression or classification. XGBoost is an implementation of gradient boosted decision trees designed for speed and performance. #XGBoost Classification model from xgboost import XGBClassifier # instantiate the model xgb = XGBClassifier(learning_rate=0.4,max_depth=7) #fit the model xgb.fit(X_train, y_train) XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1, colsample_bynode=1, colsample_bytree=1, gamma=0, learning_rate=0.4, max_delta_step=0, max_depth=7, min_child_weight=1, missing=None, n_estimators=100, n_jobs=1, nthread=None, objective='binary:logistic', random_state=0, reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=None, silent=None, subsample=1, verbosity=1) #predicting the target value from the model for the samples y_test_xgb = xgb.predict(X_test) y_train_xgb = xgb.predict(X_train) Performance Evaluation: #computing the accuracy of the model performance acc_train_xgb = accuracy_score(y_train,y_train_xgb) acc_test_xgb = accuracy_score(y_test,y_test_xgb) print("XGBoost: Accuracy on training Data: {:.3f}".format(acc_train_xgb)) print("XGBoost : Accuracy on test Data: {:.3f}".format(acc_test_xgb)) XGBoost: Accuracy on training Data: 0.866 XGBoost : Accuracy on test Data: 0.864 Storing the results: #storing the results. The below mentioned order of parameter passing is important. #Caution: Execute only once to avoid duplications. storeResults('XGBoost', acc_train_xgb, acc_test_xgb) 7.5. Autoencoder Neural Network An auto encoder is a neural network that has the same number of input neurons as it does outputs. The hidden layers of the neural network will have fewer neurons than the input/output neurons. Because there are fewer neurons, the auto-encoder must learn to encode the input to the fewer hidden neurons. The predictors (x) and output (y) are exactly the same in an auto encoder. #importing required packages import keras from keras.layers import Input, Dense from keras import regularizers import tensorflow as tf from keras.models import Model from sklearn import metrics Using TensorFlow backend. #building autoencoder model input_dim = X_train.shape[1] encoding_dim = input_dim input_layer = Input(shape=(input_dim, )) encoder = Dense(encoding_dim, activation="relu", activity_regularizer=regularizers.l1(10e-4))(input_layer) encoder = Dense(int(encoding_dim), activation="relu")(encoder) encoder = Dense(int(encoding_dim-2), activation="relu")(encoder) code = Dense(int(encoding_dim-4), activation='relu')(encoder) decoder = Dense(int(encoding_dim-2), activation='relu')(code) decoder = Dense(int(encoding_dim), activation='relu')(encoder) decoder = Dense(input_dim, activation='relu')(decoder) autoencoder = Model(inputs=input_layer, outputs=decoder) autoencoder.summary() Model: "model_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) (None, 16) 0 _________________________________________________________________ dense_1 (Dense) (None, 16) 272 _________________________________________________________________ dense_2 (Dense) (None, 16) 272 _________________________________________________________________ dense_3 (Dense) (None, 14) 238 _________________________________________________________________ dense_6 (Dense) (None, 16) 240 _________________________________________________________________ dense_7 (Dense) (None, 16) 272 ================================================================= Total params: 1,294 Trainable params: 1,294 Non-trainable params: 0 _________________________________________________________________ #compiling the model autoencoder.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) #Training the model history = autoencoder.fit(X_train, X_train, epochs=10, batch_size=64, shuffle=True, validation_split=0.2) Train on 6400 samples, validate on 1600 samples Epoch 1/10 6400/6400 [==============================] - 0s 51us/step - loss: 1.3997 - accuracy: 0.7132 - val_loss: -0.3941 - val_accuracy: 0.7890 Epoch 2/10 6400/6400 [==============================] - 0s 24us/step - loss: -0.4269 - accuracy: 0.7821 - val_loss: -0.5190 - val_accuracy: 0.7812 Epoch 3/10 6400/6400 [==============================] - 0s 24us/step - loss: -1.0514 - accuracy: 0.7908 - val_loss: -1.3147 - val_accuracy: 0.8149 Epoch 4/10 6400/6400 [==============================] - 0s 24us/step - loss: -1.3118 - accuracy: 0.8200 - val_loss: -1.3532 - val_accuracy: 0.8128 Epoch 5/10 6400/6400 [==============================] - 0s 25us/step - loss: -1.3789 - accuracy: 0.8168 - val_loss: -1.4710 - val_accuracy: 0.8190 Epoch 6/10 6400/6400 [==============================] - 0s 25us/step - loss: -1.4435 - accuracy: 0.8187 - val_loss: -1.5160 - val_accuracy: 0.8204 Epoch 7/10 6400/6400 [==============================] - 0s 25us/step - loss: -1.4951 - accuracy: 0.8215 - val_loss: -1.5601 - val_accuracy: 0.8240 Epoch 8/10 6400/6400 [==============================] - 0s 23us/step - loss: -1.5208 - accuracy: 0.8192 - val_loss: -1.5912 - val_accuracy: 0.8236 Epoch 9/10 6400/6400 [==============================] - 0s 25us/step - loss: -1.5044 - accuracy: 0.8140 - val_loss: -1.5868 - val_accuracy: 0.8191 Epoch 10/10 6400/6400 [==============================] - 0s 25us/step - loss: -1.5554 - accuracy: 0.8214 - val_loss: -1.6153 - val_accuracy: 0.8205 Performance Evaluation: acc_train_auto = autoencoder.evaluate(X_train, X_train)[1] acc_test_auto = autoencoder.evaluate(X_test, X_test)[1] print('\nAutoencoder: Accuracy on training Data: {:.3f}' .format(acc_train_auto)) print('Autoencoder: Accuracy on test Data: {:.3f}' .format(acc_test_auto)) 8000/8000 [==============================] - 0s 18us/step 2000/2000 [==============================] - 0s 20us/step Autoencoder: Accuracy on training Data: 0.819 Autoencoder: Accuracy on test Data: 0.818 Storing the results: #storing the results. The below mentioned order of parameter passing is important. #Caution: Execute only once to avoid duplications. storeResults('AutoEncoder', acc_train_auto, acc_test_auto) 7.6. Support Vector Machines In machine learning, support-vector machines (SVMs, also support-vector networks) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one or the other of two categories, an SVM training algorithm builds a model that assigns new examples to one category or the other, making it a non-probabilistic binary linear classifier. #Support vector machine model from sklearn.svm import SVC # instantiate the model svm = SVC(kernel='linear', C=1.0, random_state=12) #fit the model svm.fit(X_train, y_train) SVC(C=1.0, break_ties=False, cache_size=200, class_weight=None, coef0=0.0, decision_function_shape='ovr', degree=3, gamma='scale', kernel='linear', max_iter=-1, probability=False, random_state=12, shrinking=True, tol=0.001, verbose=False) #predicting the target value from the model for the samples y_test_svm = svm.predict(X_test) y_train_svm = svm.predict(X_train) Performance Evaluation: #computing the accuracy of the model performance acc_train_svm = accuracy_score(y_train,y_train_svm) acc_test_svm = accuracy_score(y_test,y_test_svm) print("SVM: Accuracy on training Data: {:.3f}".format(acc_train_svm)) print("SVM : Accuracy on test Data: {:.3f}".format(acc_test_svm)) SVM: Accuracy on training Data: 0.798 SVM : Accuracy on test Data: 0.818 Storing the results: #storing the results. The below mentioned order of parameter passing is important. #Caution: Execute only once to avoid duplications. storeResults('SVM', acc_train_svm, acc_test_svm) 8. Comparision of Models To compare the models performance, a dataframe is created. The columns of this dataframe are the lists created to store the results of the model. #creating dataframe results = pd.DataFrame({ 'ML Model': ML_Model, 'Train Accuracy': acc_train, 'Test Accuracy': acc_test}) results ML Model Train Accuracy Test Accuracy 0 Decision Tree 0.810 0.826 1 Random Forest 0.814 0.834 2 Multilayer Perceptrons 0.858 0.863 3 XGBoost 0.866 0.864 4 AutoEncoder 0.819 0.818 5 SVM 0.798 0.818 #Sorting the datafram on accuracy results.sort_values(by=['Test Accuracy', 'Train Accuracy'], ascending=False) ML Model Train Accuracy Test Accuracy 3 XGBoost 0.866 0.864 2 Multilayer Perceptrons 0.858 0.863 1 Random Forest 0.814 0.834 0 Decision Tree 0.810 0.826 4 AutoEncoder 0.819 0.818 5 SVM 0.798 0.818 For the above comparision, it is clear that the XGBoost Classifier works well with this dataset. So, saving the model for future use. # save XGBoost model to file import pickle pickle.dump(xgb, open("XGBoostClassifier.pickle.dat", "wb")) Testing the saved model: # load model from file loaded_model = pickle.load(open("XGBoostClassifier.pickle.dat", "rb")) loaded_model XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1, colsample_bynode=1, colsample_bytree=1, gamma=0, learning_rate=0.4, max_delta_step=0, max_depth=7, min_child_weight=1, missing=nan, n_estimators=100, n_jobs=1, nthread=None, objective='binary:logistic', random_state=0, reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=None, silent=None, subsample=1, verbosity=1)