Eli Weinstock-Herman

Easier Boundary Testing: Keep Parse/Validation/Format rules out of your HTML View

Original post posted on October 24, 2016 at LessThanDot.com

In a typical single-page application, type and validation logic is entered in the HTML view and we rely on our binding framework or a validation library to layer this behavior onto the form. There are trade-offs to this approach, which are mostly negative as you get into larger, longer-lived applications.

When we embed validation rules and logic into the View, we’re mostly limited to UI Testing to validate them (the most costly layer of the testing pyramid). We also pay an ongoing tax of having to re-define rules every place we put an input field (how do we format percentages for this field? ah, the same way as the other 100 places). Some frameworks even pass invalid or undefined values through to the underlying code, forcing us to build extra layers of defensive logic throughout the codebase.

Meanwhile, testing below the UI means we can do this:

Testing Model Inputs

Boundary Testing at Unit Test Speeds

In MVVM Validation with KnockoutJS – Don’t put it in the View/HTML, I created a small, artificial codebase to show a method of embedding user input logic in code rather than HTML attributes. Some of the value to this approach includes being able to define a domain-specific set of formats/validation for your application, access to the read/write pipelines to add extra behavior or transformations, freedom to assume only good values make it into your business logic, and the ability to write basic unit tests to verify your application behavior for good and bad inputs.

Today we’ll explore one of the opportunities we left the door open to: whether validation rules like min/max are actually working correctly.

Code for this post is available here: github.com/tarwn/Blog_KnockoutMVVMPatterns/**/validationWithTests

Boundary Testing

Boundary testing is testing the extreme edges of what is valid for our inputs, for instance what happens just inside the minimum value, at the minimum value, and just outside. Fuzz testing (which we’ll look at in a later post) is a technique that tests what happens when random, unexpected inputs occur. There are entire suites of tools built to do both of these things, but with our input logic abstracted away from the HTML, we can perform a level of these tests from unit tests code and get a large amount of the value we would get from a more costly UI testing setup at a fraction of the cost.

Let’s look at the basic sample form from the prior post:

Formatted and Validated Inputs

Formatted and Validated Inputs

The binding for each of these inputs binds to a userInput extension using knockoutjs. The user input is defined as an extension of the underlying observable value, with a defined type, optional boundaries, and the source observable we are extending that is only set when the input is valid.

Here is the PresentationModel that wraps around our data Model and defines how we present the model to a human:

Javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

function OrderLinePresentationModel(orderLine){
    var self = this;
 
    self.model = orderLine;
 
    self.name = orderLine.name.extend({ validate: { type: stringType, min: 1, max: 25, required: true } });
    self.quantity = orderLine.quantity.extend({ validate: { type: integerType, min: 1, max: 500, required: true } });
    self.price = orderLine.price.extend({ validate: { type: currencyType, min: 0, max: 100, required: true } });
 
    self.total = ko.pureComputed(function(){
        return currencyType.format(orderLine.total());
    });
 
    self.isValid = ko.pureComputed(function(){
        return !self.name.validation().isError() &&
            !self.quantity.validation().isError() &&
            !self.price.validation().isError();
    });
}
function OrderLinePresentationModel(orderLine){
	var self = this;

	self.model = orderLine;

	self.name = orderLine.name.extend({ validate: { type: stringType, min: 1, max: 25, required: true } });
	self.quantity = orderLine.quantity.extend({ validate: { type: integerType, min: 1, max: 500, required: true } });
	self.price = orderLine.price.extend({ validate: { type: currencyType, min: 0, max: 100, required: true } });

	self.total = ko.pureComputed(function(){
		return currencyType.format(orderLine.total());
	});

	self.isValid = ko.pureComputed(function(){
		return !self.name.validation().isError() &&
			!self.quantity.validation().isError() &&
			!self.price.validation().isError();
	});
}

In English, we are exposing the following properties from the underlying OrderLine Model:

  • Name – a string type that must have 1-25 characters
  • Quantity – an integer type that must be between 1 and 500
  • Price – a Currency type that must be between $0 and $100
  • Total – total for the line that is formatted as a Currency type
  • IsValid – A property that reflects whether all of the inputs have valid values

The “type” objects are used during the read/write pipelines:

Presentation Model - Read/Write Pipelines

Presentation Model – Read/Write Pipelines

  • On Read: Run the raw Model value through the Formatter to produce output
  • On Write/Input: Run the input through tryParse and then tryValidate (and then custom validation properties), only writing it to the underlying Model property if it passes both steps

Here is what one of those “type” objects looks like as a literal:

Javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

return {
    emptyValue: null,
    format: function(value){
        if(value == null){
            return '';
        }
        else{
            return value.toLocaleString('en-US', {
                style: 'currency',
                currency: 'USD',
                currencyDisplay: 'symbol',
                useGrouping: true
            });
        }
    },
    tryParse: function(value){
        // strip out commas and $
        var parsedResult = parseFloat(value.replace(/[\$,]/g,''));
        if(isNaN(parsedResult)){
            return inputResult.failedInput("'" + value + "' is not a valid currency value");
        }
        else{
            return inputResult.successfulInput(parsedResult);
        }
    },
    tryValidate: function(value, options){
        if(options.min != undefined && value < options.min){
            return inputResult.failedInput("'" + value + "' is less than the supported minimum of '" + options.min + "'");
        }
 
        if(options.max != undefined && value > options.max){
            return inputResult.failedInput("'" + value + "' is greater than the supported maximum of '" + options.max + "'");
        }
 
        return inputResult.successfulInput(value);      
    }
};
return {
	emptyValue: null,
	format: function(value){
		if(value == null){
			return '';
		}
		else{
			return value.toLocaleString('en-US', {
				style: 'currency',
				currency: 'USD',
				currencyDisplay: 'symbol',
				useGrouping: true
			});
		}
	},
	tryParse: function(value){
		// strip out commas and $
		var parsedResult = parseFloat(value.replace(/[\$,]/g,''));
		if(isNaN(parsedResult)){
			return inputResult.failedInput("'" + value + "' is not a valid currency value");
		}
		else{
			return inputResult.successfulInput(parsedResult);
		}
	},
	tryValidate: function(value, options){
		if(options.min != undefined && value < options.min){
			return inputResult.failedInput("'" + value + "' is less than the supported minimum of '" + options.min + "'");
		}

		if(options.max != undefined && value > options.max){
			return inputResult.failedInput("'" + value + "' is greater than the supported maximum of '" + options.max + "'");
		}

		return inputResult.successfulInput(value);		
	}
};

Using a knockout extension, we can ensure all writes pass through the parse and validate logic before being written to the true underlying model, surface an error state if either fails, and display a formatted value independent of the raw value in that underlying model.

Adding Boundary Tests

So we have two options for boundary testing, we can test each available “type” with some common boundary settings or we can test each PresentationModel’s inputs with the defined ones. The intent of the test is to make sure we handle boundaries the way we think we do, so I lean towards the more target “type” tests as getting the most bang for the buck.

Testing Model Inputs

Testing Model Inputs

Here is sample code for validating the boundaries on the Order Line, which has a text, integer, and currency input:
specs/models/orderLinePresentationModel.boundary.spec.js

Javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

describe('boundary tests', function(){
 
    var testCases = [
        { name: 'name input - shorter than min', field: 'name', input: '', isError: true },
        { name: 'name input - longer than min', field: 'name', input: '1', isError: false },
        { name: 'name input - shorter than max', field: 'name', input: '123456789012345678901234', isError: false },
        { name: 'name input - max', field: 'name', input: '1234567890123456789012345', isError: false },
        { name: 'name input - longer than max', field: 'name', input: '12345678901234567890123456', isError: true },
 
        { name: 'quantity input - less than min', field: 'quantity', input: '-1', isError: true },
        { name: 'quantity input - min', field: 'quantity', input: '1', isError: false },
        { name: 'quantity input - above min', field: 'quantity', input: '2', isError: false },
        { name: 'quantity input - below max', field: 'quantity', input: '499', isError: false },
        { name: 'quantity input - max', field: 'quantity', input: '500', isError: false },
        { name: 'quantity input - above max', field: 'quantity', input: '501', isError: true },
 
        { name: 'price input - less than min', field: 'price', input: '-1', isError: true },
        { name: 'price input - min', field: 'price', input: '1', isError: false },
        { name: 'price input - above min', field: 'price', input: '2', isError: false },
        { name: 'price input - below max', field: 'price', input: '99', isError: false },
        { name: 'price input - max', field: 'price', input: '100', isError: false },
        { name: 'price input - above max', field: 'price', input: '101', isError: true },
 
    ];
 
    testCases.forEach(function(testCase){
        var testName = testCase.name + ': is a ' + (testCase.isError ? 'validation error' : 'successful input');
 
        it(testName, function(){
            var testLine = new OrderLineModel({});
            var testLineP = new OrderLinePresentationModel(testLine);
            var inputUnderTest = testLineP[testCase.field];
 
            inputUnderTest(testCase.input);             
 
            expect(inputUnderTest.validation().isError()).toBe(testCase.isError);
        });
 
    });
 
});
describe('boundary tests', function(){

	var testCases = [
		{ name: 'name input - shorter than min', field: 'name', input: '', isError: true },
		{ name: 'name input - longer than min', field: 'name', input: '1', isError: false },
		{ name: 'name input - shorter than max', field: 'name', input: '123456789012345678901234', isError: false },
		{ name: 'name input - max', field: 'name', input: '1234567890123456789012345', isError: false },
		{ name: 'name input - longer than max', field: 'name', input: '12345678901234567890123456', isError: true },

		{ name: 'quantity input - less than min', field: 'quantity', input: '-1', isError: true },
		{ name: 'quantity input - min', field: 'quantity', input: '1', isError: false },
		{ name: 'quantity input - above min', field: 'quantity', input: '2', isError: false },
		{ name: 'quantity input - below max', field: 'quantity', input: '499', isError: false },
		{ name: 'quantity input - max', field: 'quantity', input: '500', isError: false },
		{ name: 'quantity input - above max', field: 'quantity', input: '501', isError: true },

		{ name: 'price input - less than min', field: 'price', input: '-1', isError: true },
		{ name: 'price input - min', field: 'price', input: '1', isError: false },
		{ name: 'price input - above min', field: 'price', input: '2', isError: false },
		{ name: 'price input - below max', field: 'price', input: '99', isError: false },
		{ name: 'price input - max', field: 'price', input: '100', isError: false },
		{ name: 'price input - above max', field: 'price', input: '101', isError: true },

	];

	testCases.forEach(function(testCase){
		var testName = testCase.name + ': is a ' + (testCase.isError ? 'validation error' : 'successful input');

		it(testName, function(){
			var testLine = new OrderLineModel({});
			var testLineP = new OrderLinePresentationModel(testLine);
			var inputUnderTest = testLineP[testCase.field];

			inputUnderTest(testCase.input);				

			expect(inputUnderTest.validation().isError()).toBe(testCase.isError);
		});

	});

});

So far not a lot of duplication of effort, but in a larger application we would expect to have 10s or 100s of integer, text, currency, date, etc inputs. So the effort to test at this Model level doesn’t seem to add much value over testing directly at the type level, unless we’re passing in custom validation that would cause a specific input to operate differently than others of a similar type.

Testing Input Types

Testing Input Types

Here’s how we can instead do boundary testing for the more generalized input type level:
specs/inputTypes/all.boundary.spec.js

Javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

    describe('boundary tests', function(){
    
        var testCases = [
            { name: 'currency input - less than min',   options: { type: currencyType, min: 0, max: 10 }, input: '-.5', isError: true },
            { name: 'currency input - min length',      options: { type: currencyType, min: 0, max: 10 }, input: '0', isError: false },
            { name: 'currency input - more than min',   options: { type: currencyType, min: 0, max: 10 }, input: '2.5', isError: false },
            { name: 'currency input - less than max',   options: { type: currencyType, min: 0, max: 10 }, input: '7.5', isError: false },
            { name: 'currency input - max',             options: { type: currencyType, min: 0, max: 10 }, input: '10', isError: false },
            { name: 'currency input - more than max',   options: { type: currencyType, min: 0, max: 10 }, input: '10.5', isError: true },
 
            …
            
            { name: 'string input - longer than max',   options: { type: stringType, min: 5, max: 8 }, input: '123456789', isError: true },
 
        ];
 
        testCases.forEach(function(testCase){
            var testName = testCase.name + ': is a ' + (testCase.isError ? 'validation error' : 'successful input');
 
            it(testName, function(){
                var testField = ko.observable();
                var presentableTestField = testField.extend({ validate: testCase.options });
 
                presentableTestField(testCase.input);
 
                expect(presentableTestField.validation().isError()).toBe(testCase.isError);
            });
        });
 
    });
	describe('boundary tests', function(){
	
		var testCases = [
			{ name: 'currency input - less than min',	options: { type: currencyType, min: 0, max: 10 }, input: '-.5', isError: true },
			{ name: 'currency input - min length',		options: { type: currencyType, min: 0, max: 10 }, input: '0', isError: false },
			{ name: 'currency input - more than min',	options: { type: currencyType, min: 0, max: 10 }, input: '2.5', isError: false },
			{ name: 'currency input - less than max',	options: { type: currencyType, min: 0, max: 10 }, input: '7.5', isError: false },
			{ name: 'currency input - max',				options: { type: currencyType, min: 0, max: 10 }, input: '10', isError: false },
			{ name: 'currency input - more than max',	options: { type: currencyType, min: 0, max: 10 }, input: '10.5', isError: true },

			…
			
			{ name: 'string input - longer than max',	options: { type: stringType, min: 5, max: 8 }, input: '123456789', isError: true },

		];

		testCases.forEach(function(testCase){
			var testName = testCase.name + ': is a ' + (testCase.isError ? 'validation error' : 'successful input');

			it(testName, function(){
				var testField = ko.observable();
				var presentableTestField = testField.extend({ validate: testCase.options });

				presentableTestField(testCase.input);

				expect(presentableTestField.validation().isError()).toBe(testCase.isError);
			});
		});

	});

We have a common component that handles all input logic that is governed by each type, so now our tests focus just on the collision of the types and input logic. Rather than require our teammates to enter tests every time they re-use a type for a new field in a completely expected way, we now only need to add tests when they want to add in some custom validation specific to that field, getting the value from the boundary testing while minimizing the cost.

But that’s not real user input?

That’s true, it isn’t. And since we already put some thought into our abstraction of the user input logic, it’s possible this won’t catch any errors that a unit test would miss. On the other hand, had we included the validation and formatting as attributes in the HTML, not only would we have a lot of duplication of effort, this would have required us to pull out our UI automation frameworks, with all the performance and ongoing maintenance costs that assumes. Instead, we can implement boundary testing fairly cheaply even if we thin it might be overkill, and let it run with our unit test suite at unit test speeds. The only gap that real user inputs would add, at far higher cost, is whether we had bound the UI component correctly to the input behind the scenes, which feels like another class of problem which would have effect reaching far beyond boundary conditions and probably should be tested on it’s own.

Comments are available on the original post at lessthandot.com