MOBISIG vs. MCYT

July 23th 2017

Databases

Table 2: Basic characteristics of the MCYT and MOBISIG databases

	MCYT	MOBISIG
Device	WACOM, INTUOS A6 USB.	Nexus 9 tablet, Android 6.0
Frequency	100 Hz (periodic sampling)	Approx. 60 Hz (event based sampling)
Coordinates	x: 0-12700, y: 0-9700	x: 0-1536, y: 0-2048
Used raw data	x(t), y(t), p(t)	x(t), y(t), p(t)
Sessions		3
Subjects	100 (44 women, 56 men)	83 (34 women, 49 men)
Samples		Sess1: 15 genuine Sess2: 15 genuine, 10 forgery Sess3: 15 genuine, 10 forgery
URL	MCYT https://atvs.ii.uam.es/atvs/mcyt100s.html	MOBISIG https://www.ms.sapientia.ro/~manyi/mobisig.html

Features

Local features

Figure 1 Local features (time-functions) for a genuine signature from the MOBISIG database.

Global features

	State-of-the-art	Information theory
1	Signature total duration	Entropy of x(t)
2	Average velocity	Entropy of y(t)
3	Average pressure	Entropy of p(t)
4	Max velocity ()	Fisher of x(t)
5	Average pointwise velocity	Fisher of y(t)
6	Average pointwise acceleration	Fisher of p(t)
7	Sign changes of	Entropy of
8	Sign changes of	Entropy of
9	Sign changes of	Entropy of
10	Sign changes of	Fisher of
11	Average	Fisher of
12	Average	Fisher of
13	Sign changes of
14	Sign changes of
15	Number of sampled points

Summary

	Skilled Forgery
	MOBISIG			MCYT
	EERg	EERu(mean)	EERu(stdev)	EERg	EERu(mean)	EERu(stdev)
(1) Inf. Theory (12)	20.69	17.80	11.94	14.12	10.80	9.17
(2) State-of-the-art (15)	11.08	8.16	8.48	6.68	4.03	6.40
(1)+(2) Both (27)	10.72	8.22	8.71	5.79	3.81	5.79
	Random Forgery
	MOBISIG			MCYT
(1) Inf. Theory (12)	11.62	9.49	6.18	10.31	7.24	6.51
(2) State-of-the-art (15)	5.83	3.77	4.74	3.69	1.66	3.53
(1)+(2) Both (27)	4.62	2.56	3.82	3.13	1.51	3.45

GLOBAL SYSTEM

Skilled Forgery (SF)

EvType	Database	#numFeatures	#numTraining	#numPositive	#numNegative	Detector	EERg	EERu(mean)	EERu(stdev)
SF	MOBISIG	27	5	20	20	EUCLIDEAN	20.15	16.33	14.34
SF	MOBISIG	27	5	20	20	MANHATTAN	18.77	15	13.38
SF	MOBISIG	27	5	20	20	KNN	19.31	15.87	13.16
SF	MOBISIG	27	10	15	20	EUCLIDEAN	13.32	9.95	9.63
SF	MOBISIG	27	10	15	20	MANHATTAN	14.32	11.3	10.93
SF	MOBISIG	27	10	15	20	KNN	13.86	10.2	9.84
SF	MOBISIG	27	15	10	20	EUCLIDEAN	10.72	8.22	8.71
SF	MOBISIG	27	15	10	20	MANHATTAN	12.41	9.55	9.64
SF	MOBISIG	27	15	10	20	KNN	12.2	8.73	8.88
SF	MOBISIG	15	5	20	20	EUCLIDEAN	20.03	15.93	13.94
SF	MOBISIG	15	5	20	20	MANHATTAN	18.86	14.67	12.21
SF	MOBISIG	15	5	20	20	KNN	19.7	15.54	13.41
SF	MOBISIG	15	10	15	20	EUCLIDEAN	13.81	10.67	10.45
SF	MOBISIG	15	10	15	20	MANHATTAN	14.87	12	11
SF	MOBISIG	15	10	15	20	KNN	13.77	10.76	9.92
SF	MOBISIG	15	15	10	20	EUCLIDEAN	11.08	8.16	8.48
SF	MOBISIG	15	15	10	20	MANHATTAN	12.62	9.94	9.73
SF	MOBISIG	15	15	10	20	KNN	11.63	9.1	8.5
SF	MOBISIG	12	5	20	20	EUCLIDEAN	27.71	24.31	14.3
SF	MOBISIG	12	5	20	20	MANHATTAN	27.86	24.34	14.03
SF	MOBISIG	12	5	20	20	KNN	26.9	24.07	14.52
SF	MOBISIG	12	10	15	20	EUCLIDEAN	22.67	19.87	11.98
SF	MOBISIG	12	10	15	20	MANHATTAN	23.34	20.95	12.45
SF	MOBISIG	12	10	15	20	KNN	24.05	21.56	13.51
SF	MOBISIG	12	15	10	20	EUCLIDEAN	20.69	17.8	11.94
SF	MOBISIG	12	15	10	20	MANHATTAN	21.6	18.83	12.06
SF	MOBISIG	12	15	10	20	KNN	22.29	18.73	14.04
SF	MCYT	27	5	20	25	EUCLIDEAN	12.44	8.89	10.13
SF	MCYT	27	5	20	25	MANHATTAN	11.85	7.83	8.44
SF	MCYT	27	5	20	25	KNN	12.5	8.94	9.87
SF	MCYT	27	10	15	25	EUCLIDEAN	7.27	4.48	5.73
SF	MCYT	27	10	15	25	MANHATTAN	7.65	4.65	5.48
SF	MCYT	27	10	15	25	KNN	8.85	5.54	7.93
SF	MCYT	27	15	10	25	EUCLIDEAN	5.79	3.81	5.79
SF	MCYT	27	15	10	25	MANHATTAN	6.19	4.27	6.1
SF	MCYT	27	15	10	25	KNN	6.22	4.33	6.04
SF	MCYT	15	5	20	25	EUCLIDEAN	13.96	8.87	10.43
SF	MCYT	15	5	20	25	MANHATTAN	13.54	8.24	9.24
SF	MCYT	15	5	20	25	KNN	13.83	9.13	10.63
SF	MCYT	15	10	15	25	EUCLIDEAN	7.9	4.67	5.86
SF	MCYT	15	10	15	25	MANHATTAN	8.35	5.22	6.24
SF	MCYT	15	10	15	25	KNN	8.59	5.72	7.62
SF	MCYT	15	15	10	25	EUCLIDEAN	6.68	4.03	6.4
SF	MCYT	15	15	10	25	MANHATTAN	7.13	4.3	5.99
SF	MCYT	15	15	10	25	KNN	7.47	4.56	6.63
SF	MCYT	12	5	20	25	EUCLIDEAN	21.3	16.91	11.56
SF	MCYT	12	5	20	25	MANHATTAN	20.29	15.43	10.21
SF	MCYT	12	5	20	25	KNN	21.79	18.51	12.92
SF	MCYT	12	10	15	25	EUCLIDEAN	16.73	12.75	9.47
SF	MCYT	12	10	15	25	MANHATTAN	16.11	13.03	8.97
SF	MCYT	12	10	15	25	KNN	19.82	15.68	11.81
SF	MCYT	12	15	10	25	EUCLIDEAN	14.12	10.8	9.17
SF	MCYT	12	15	10	25	MANHATTAN	14.48	11.87	9.1
SF	MCYT	12	15	10	25	KNN	17.65	15.29	12.5

Random Forgery (RF)

EvType	Database	#numFeatures	#numTraining	#numPositive	#numNegative	Detector	EERg	EERu(mean)	EERu(stdev)
RF	MOBISIG	27	5	20	82	EUCLIDEAN	12.44	7.67	7.8
RF	MOBISIG	27	5	20	82	MANHATTAN	11.02	5.78	4.91
RF	MOBISIG	27	5	20	82	KNN	12.18	7.87	8.23
RF	MOBISIG	27	10	15	82	EUCLIDEAN	5.59	3.48	3.72
RF	MOBISIG	27	10	15	82	MANHATTAN	6.01	3.94	4.14
RF	MOBISIG	27	10	15	82	KNN	6.65	3.83	4.28
RF	MOBISIG	27	15	10	82	EUCLIDEAN	4.62	2.56	3.82
RF	MOBISIG	27	15	10	82	MANHATTAN	5.18	3.13	4.34
RF	MOBISIG	27	15	10	82	KNN	5.3	3.05	4.18
RF	MOBISIG	15	5	20	82	EUCLIDEAN	14.67	9.46	11.07
RF	MOBISIG	15	5	20	82	MANHATTAN	13.5	7.7	6.7
RF	MOBISIG	15	5	20	82	KNN	14.74	9.78	11.05
RF	MOBISIG	15	10	15	82	EUCLIDEAN	7.35	4.7	4.42
RF	MOBISIG	15	10	15	82	MANHATTAN	8.14	5.44	4.39
RF	MOBISIG	15	10	15	82	KNN	7.63	4.94	5.1
RF	MOBISIG	15	15	10	82	EUCLIDEAN	5.83	3.77	4.74
RF	MOBISIG	15	15	10	82	MANHATTAN	6.98	4.88	5.34
RF	MOBISIG	15	15	10	82	KNN	6.34	4.43	5.24
RF	MOBISIG	12	5	20	82	EUCLIDEAN	17.91	13.38	8.49
RF	MOBISIG	12	5	20	82	MANHATTAN	17.5	13.9	8.99
RF	MOBISIG	12	5	20	82	KNN	18.99	15.02	9.9
RF	MOBISIG	12	10	15	82	EUCLIDEAN	12.85	10.26	6.25
RF	MOBISIG	12	10	15	82	MANHATTAN	13.17	11.47	6.85
RF	MOBISIG	12	10	15	82	KNN	15.09	12.13	7.27
RF	MOBISIG	12	15	10	82	EUCLIDEAN	11.62	9.49	6.18
RF	MOBISIG	12	15	10	82	MANHATTAN	11.59	10.44	6.97
RF	MOBISIG	12	15	10	82	KNN	13.61	10.6	7.18
RF	MCYT	27	5	20	99	EUCLIDEAN	8.37	4.05	4.63
RF	MCYT	27	5	20	99	MANHATTAN	7.2	3.56	4.16
RF	MCYT	27	5	20	99	KNN	8.3	4.21	4.71
RF	MCYT	27	10	15	99	EUCLIDEAN	3.94	2.01	3.17
RF	MCYT	27	10	15	99	MANHATTAN	3.93	2.19	3.48
RF	MCYT	27	10	15	99	KNN	4.26	2.55	3.98
RF	MCYT	27	15	10	99	EUCLIDEAN	3.13	1.51	3.45
RF	MCYT	27	15	10	99	MANHATTAN	3.35	1.76	3.47
RF	MCYT	27	15	10	99	KNN	3.63	1.69	4.17
RF	MCYT	15	5	20	99	EUCLIDEAN	9.14	4.71	5.47
RF	MCYT	15	5	20	99	MANHATTAN	8.01	4.42	5.02
RF	MCYT	15	5	20	99	KNN	8.93	4.8	5.66
RF	MCYT	15	10	15	99	EUCLIDEAN	4.15	2.52	3.33
RF	MCYT	15	10	15	99	MANHATTAN	4.52	2.72	3.41
RF	MCYT	15	10	15	99	KNN	4.89	2.85	3.83
RF	MCYT	15	15	10	99	EUCLIDEAN	3.69	1.66	3.53
RF	MCYT	15	15	10	99	MANHATTAN	3.69	1.82	3.32
RF	MCYT	15	15	10	99	KNN	3.89	2.39	4.78
RF	MCYT	12	5	20	99	EUCLIDEAN	17.2	11.72	8.08
RF	MCYT	12	5	20	99	MANHATTAN	15.68	10.75	7.3
RF	MCYT	12	5	20	99	KNN	17.39	12.22	8.3
RF	MCYT	12	10	15	99	EUCLIDEAN	12.28	8.36	6.45
RF	MCYT	12	10	15	99	MANHATTAN	12.01	8.86	7.15
RF	MCYT	12	10	15	99	KNN	14.18	10.24	8.23
RF	MCYT	12	15	10	99	EUCLIDEAN	10.31	7.24	6.51
RF	MCYT	12	15	10	99	MANHATTAN	10.44	8.14	7.36
RF	MCYT	12	15	10	99	KNN	13.88	9.38	8.84

LOCAL SYSTEM

MCYT

· Training: 5, 10, 15

· Test - skilled:

o Positive: 25 – numTrSamples= 20, 15, 10

o Negative: 25 samples

· Test – random

o Positive: 25 – numTrSamples= 20, 15, 10

o Negative: 99 samples (1 sample from each other users)

MCYT	Skilled				Random
#trSamples		EERg	EERu(mean)	EERu(stdev)		EERg	EERu(mean)	EERu(stdev)
5	xy	9.93	4.14	6.41	xy	0.82	0.17	0.87
	x1y1	8.03	2.22	6.18	x1y1	0.75	0.1	0.57
	x2y2	7.96	2.74	4.32	x2y2	1.62	0.31	1.23
	xyx1y1x2y2	4.52	1.65	3.99	xyx1y1x2y2	0.57	0.14	0.78
	xyx1y1x2y2pp1	3.89	1.40	3.59	xyx1y1x2y2pp1	0.29	0.09	0.64
10	xy	7.99	2.99	5.33	xy	0.59	0.21	1.13
	x1y1	6.95	1.60	4.26	x1y1	0.63	0.04	0.22
	x2y2	7.13	2.18	3.83	x2y2	1.19	0.20	0.99
	xyx1y1x2y2	3.77	1.15	3.52	xyx1y1x2y2	0.49	0.06	0.49
	xyx1y1x2y2pp1	3.35	1.25	3.85	xyx1y1x2y2pp1	0.37	0.02	0.15
15	xy	7.45	2.53	5.02	xy	0.55	0.11	1.01
	x1y1	6.31	1.01	3.88	x1y1	0.59	0.03	0.18
	x2y2	6.69	1.61	3.78	x2y2	1.06	0.07	0.37
	xyx1y1x2y2	3.40	0.91	3.78	xyx1y1x2y2	0.45	0.02	0.15
	xyx1y1x2y2pp1	2.97	0.93	3.72	xyx1y1x2y2pp1	0.25	0.02	0.15

MOBISIG

· Training: 5, 10, 15 (first session)

· Test - skilled:

o Positive: 15 samples (second session)

o Negative: 20 samples

· Test – random

o Positive: 15 samples (second session)

o Negative: 82 samples (1 sample from each other users)

MOBISIG	Skilled				Random
#trSamples		EERg	EERu(mean)	EERu(stdev)		EERg	EERu(mean)	EERu(stdev)
5	xy	24.45	13.58	14.43	xy	2.09	0.27	1.11
	x1y1	30.55	10.00	12.97	x1y1	1.61	0.01	0.07
	x2y2	42.97	23.36	20.38	x2y2	21.33	2.80	7.38
	xyx1y1x2y2	31.60	8.81	13.11	xyx1y1x2y2	2.95	0.02	0.11
	xyx1y1x2y2pp1	31.21	7.72	10.30	xyx1y1x2y2pp1	4.23	0.11	0.74
10	xy	21.58	11.88	13.99	xy	1.97	0.30	1.24
	x1y1	28.27	8.40	12.14	x1y1	1.44	0.01	0.07
	x2y2	42.57	20.36	18.67	x2y2	20.54	1.81	3.55
	xyx1y1x2y2	30.20	6.65	12.13	xyx1y1x2y2	2.41	0.01	0.09
	yx1y1x2y2pp1	30.00	6.57	10.05	xyx1y1x2y2pp1	4.06	0.15	0.94
15	xy	19.95	9.94	12.63	xy	1.72	0.20	1.00
	x1y1	26.70	6.43	10.26	x1y1	1.55	0.00	0.00
	x2y2	41.58	18.08	17.06	x2y2	20.24	1.52	3.36
	xyx1y1x2y2	29.31	5.63	11.37	xyx1y1x2y2	2.62	0.01	0.09
	xyx1y1x2y2pp1	28.72	5.41	9.10	xyx1y1x2y2pp1	4.15	0.07	0.63