トップに戻る

コメント (10)

jph•49分前
There's low-hanging fruit IMHO by testing using semantic naming vocabulary with semantic error vocabulary.

Example macro: assert_approx_eq_with_relative_error!(a, b, err). It's a semantic name that explains more about what it does, and the macro also provides a test failure message that guides the developer, or an AI, toward fixes and documentation.

I wrote the Rust crate Assertables to help with this. See https://crates.io/crates/assertables.

What I learned is writing semantic testing can help many more kinds of higher-value testing. As a specific example described by the article, I use enumeration testing for proving the UK National Health Service Number validator in Rust is accurate i.e. the test literally checks every number, then I can write tests such as assert_valid_nhs_number("111 111 1111").

ozim•4時間前
I would like "non-functional testing" to be called "cross-functional testing".
FabCH•2時間前
I find that the biggest problem with all these classifications is that at the end of the day, developers will write whatever works for them in the moment, and not care if it fits your theoretical box.

I did a paper on it once, to see what tests devs actually write: https://ieeexplore.ieee.org/document/11081450

Blowoffvalve•3時間前
I loved this article. I find that the most important part of engineering projects is defining a shared vocabulary for discussing and aligning with other parties.

The vocabulary proposed that clear delineates across the various concerns you're testing against is particularly helpful.

jruohonen•5時間前
I think the alleged dictum between functional and non-functional is the biggest conceptual, theoretical, or even philosophical issue.
jerf•3時間前
I think the problem with testing vocabulary is that the field is just too continuous. We have all sorts of situations in the world where there is no sharp line, yet we have terminology for various chunks of the possibility space even so because there are still some sort of clump in the space that we can identify. For example, there isn't a true sharp line between any of the primary colors, but we do not generally have a problem labelling things as "red" or "green" or whatever even so. There is a certain personality type frequently encountered online that this bothers no end, who can't really deal with the idea of words that have fuzzy boundaries around them and struggle with the concept of a "fuzzy line", but most of us get on even so.

However, when there is no natural clumping, there isn't even a place to put the fuzzy boundary necessarily. I think a lot of testing has this problem. I've disliked "unit" versus "integration" terminology for at least a decade now, because to me there's just a continuum. Yes, obviously, tests can test more or less of the code base at a time, and can draw in fewer or more dependencies in the process, and arguably there is an important distinction between 0 other dependencies included and 1, but beyond that I've thought for a long time there aren't any other distinctions worth putting a label on.

There is value to tests that mock out all the other dependencies. There is value to writing stubs. There is value to bringing up those dependencies and mocking or stubbing out their dependencies. There is value to having a full staging environment and running acceptance tests by hand, and that staging environment itself can have any number of mocks or stubs in it itself. And there is no rule that says you rigidly must make this decision based on the exact number of dependencies away something is from your code, to the extent that that is even a well-defined concept, nor are "mock" and "stub" themselves some sort of bright shining line as one can mix and match them freely, nor is that the complete list of all possibilities. The result is a fairly smooth continuum of possible tests. In my opinion, any attempt to put labels on very particular bits of it raises the risk of overprivileging certain parts of the space just because they get a name and creating the streetlight effect on the named bits and causing the others to be missed.

Even "performance testing" versus "correctness testing" gets awfully fuzzy; again, there is arguably some value in saying this test has performance concerns and this other test is just related to correctness, but even "performance testing" still has to worry about a lot of other things too and I don't find it anywhere near a binary distinction. Anyone who has spent any time with performance testing has certainly learned the risk of worrying about "performance" without also worrying about all the other things we test for at the same time, or to put it another way, I can make my performance testing look really good if I let the code it is testing be incorrect.

It is unreasonable to expect a universal nomenclature catalog of such a smooth space. We can't even as a species completely agree on color delineations in the spectrum, which is a simpler problem in most ways.

gzell•4時間前
Always enjoyable for us aged s/w engineers to see the new generations relearn the fundamentals.
g-b-r•4時間前
I recall finding different definitions in every book about testing I read.

I think most of the confusion came from people suddenly deciding that a unit test can't test less than a whole object, in OOP.

sfeizi•3時間前
does vocabulary matter anymore as coding agents can make interpretations?
gfalcao•3時間前
great infographic