 |
Matronics Email Lists
Web Forum Interface to the Matronics Email Lists
|
| View previous topic :: View next topic |
| Author |
Message |
nuckollsr(at)cox.net
Guest
|
 Posted: Sun Sep 17, 2006 6:06 am Post subject: Systems Testing |
 |
|
On 15 Sep 2006, at 10:18, Gary Casey wrote:
>
> <glcasey(at)adelphia.net>
>
> Kevin previously wrote:
>> "If you don't do the test then, someday you may
>> stumble across that condition when you hadn't planned it. If the
>> system works as expected, then everything is OK. But if the system
>> does not perform, then you may lose the aircraft.
>
> Good points, but what "assumptions" do you have in mind that should
> be tested?
| Quote: |
<peter(at)sportingaero.com>
Try writing down a list of things your fuel system must do. Write simple
statements and start with the basics - transfer fuel from the tank to
the carb/injector fuel inlet at xx pressure. Write only one 'feature'
per statement. Think about what you want to happen in unusual situations
or when components fail. Try not to think about your design. You are
creating a set of "Requirements". Include details - must be able to get
at fuel filter easily for servicing - as well as whole system issues -
system must only need servicing yearly. Define words like 'easily'.
There are now two parallel paths to follow. 1) Look at your design to
ensure you have met all of the requirements - be critical (not always
easy). If the design does not meet the requirements then change the
design.
2) Develop a verification or test for each requirement. If a ground test
is too difficult you will now understand any risks you are taking - for
example you may decide to trust the pressure gauge manufacturer's
calibration (most people do). Minimize flight testing, its risky and
getting good data is more difficult than you think. The best way is to
get someone other than the designer to write the tests (your wife may
be?), but that's not always possible.
Be honest with yourself when testing. A second pair of eyes often helps.
Try to keep it all simple.
Yours, Pete
|
Pete, thanks for posting this. An excellent illumination
of the thought processes behind setting requirements,
evaluating performance against those requirements, and
conducting a failure mode effects analysis (FMEA) from
which one (1) revises requirements and/or (2) deduces
Plan-B for each necessary item that fails to perform under
Plan-A.
With respect to development of requirements, design, testing
and failure mitigation, I would encourage Listers to publish
their thoughts here on the List. No doubt some folks will be
inclined to evaluate your words with too much emphasis on their
assessment of your lack of skill and knowledge. Expect it to happen
and know that these postings should be summarily ignored.
However, others are willing and able to assist with step/by/step
analysis of simple-ideas with the supporting logic needed for
achieving your design goals. In this manner, we elevate the
skills and knowledge of all who choose to participate.
The OBAM aviation Skunkwerks ranks amongst the largest
and most capable given the huge diversity individuals who
decide to build airplanes. They bring a wealth of knowledge
with them and there's no better place for that resource to
be cultivated and shared than right here on the List.
Bob . . .
| | - The Matronics AeroElectric-List Email Forum - | | | Use the List Feature Navigator to browse the many List utilities available such as the Email Subscriptions page, Archive Search & Download, 7-Day Browse, Chat, FAQ, Photoshare, and much more:
http://www.matronics.com/Navigator?AeroElectric-List |
|
|
|
| Back to top |
|
 |
luckymacy(at)comcast.net
Guest
|
| Posted: Sun Sep 17, 2006 6:23 am Post subject: Systems Testing |
|
|
I don't see anything wrong with the designer "writing the test", I see good things about that in the OBAM world. I'd add though to make the requirements & test plan/test procedures public so they can be "peer reviewed".
Lucky
[quote]-------------- Original message --------------
From: "Robert L. Nuckolls, III" <nuckollsr(at)cox.net>
[quote] --> AeroElectric-List message posted by: "Robert L. Nuckolls, III"
On 15 Sep 2006, at 10:18, Gary Casey wrote:
> --> AeroElectric-List message posted by: Gary Casey
>
>
> Kevin previously wrote:
>> "If you don't do the test then, someday you may
>> stumble across that condition when you hadn't planned it. If the
>> system works as expected, then everything is OK. But if the system
>> does not perform, then you may lose the aircraft.
>
> Good points, but what "assumptions" do you have in mind that should
> be tested?
>--> Ae roElec tric-List message posted by: "Peter Pengilly"
>
>
>Try writing down a list of things your fuel system must do. Write simple
>statements and start with the basics - transfer fuel from the tank to
>the carb/injector fuel inlet at xx pressure. Write only one 'feature'
>per statement. Think about what you want to happen in unusual situations
>or when components fail. Try not to think about your design. You are
>creating a set of "Requirements". Include details - must be able to get
>at fuel filter easily for servicing - as well as whole system issues -
>system must only need servicing yearly. Define words like 'easily'.
>
>There are now two parallel paths to follow. 1) Look at your design to
>ensure you have met all of the requirements - be critical (not always
>easy). If the design does not meet the requirements then change the
>design.
>
>2) Develop a verification or test for each requirement. If a ground test
>is too difficult you will now understand any risks you are taking - for
>example you may decide to trust the pressure gauge manufacturer's
>calibration (most people do). Minimize flight testing, its risky and
>getting good data is more difficult than you think. The best way is to
>get someone other than the designer to write the tests (your wife may
>be?), but that's not always possible.
>
>Be honest with yourself when testing. A second pair of eyes often helps.
>Try to keep it all simple.
>
>Yours, Pete
Pete, thanks for posting this. An excellent illumination
of the thought processes behind setting requirements,
evaluating performance against those requireme nts, a nd
conducting a failure mode effects analysis (FMEA) from
which one (1) revises requirements and/or (2) deduces
Plan-B for each necessary item that fails to perform under
Plan-A.
With respect to development of requirements, design, testing
and failure mitigation, I would encourage Listers to publish
their thoughts here on the List. No doubt some folks will be
inclined to evaluate your words with too much emphasis on their
assessment of your lack of skill and knowledge. Expect it to happen
and know that these postings should be summarily ignored.
However, others are willing and able to assist with step/by/step
analysis of simple-ideas with the supporting logic needed for
achieving your design goals. In this manner, we elevate the
skills and knowledge of all who choose to participate.
The OBAM aviation Skunkwerks ranks among .matro [quote][b]
| | - The Matronics AeroElectric-List Email Forum - | | | Use the List Feature Navigator to browse the many List utilities available such as the Email Subscriptions page, Archive Search & Download, 7-Day Browse, Chat, FAQ, Photoshare, and much more:
http://www.matronics.com/Navigator?AeroElectric-List |
|
|
|
| Back to top |
|
|
Deems Davis
Joined: 09 Jan 2006
Posts: 925
|
| Posted: Sun Sep 17, 2006 7:07 am Post subject: Systems Testing |
|
|
There's pretty compelling evidence from industry that strongly suggests
that test plans written by the designers are biased to 'prove' the
design. More often then not this is not a conscious but an unconscious
bias. Someone, who is *equally* expert on the subject who is not
invested in the design writing a test plan will uncover more errors and
uncover them earlier. Mike Fagan (at) IBM did some compelling research on
this subject and developed his 'inspection' process for software based
partly upon this principle. Other Quality guru's have done similarly on
manufacturing and other processes.
Deems Davis # 406
Panel/wiring
http://deemsrv10.com/
lucky wrote:
| Quote: | I don't see anything wrong with the designer "writing the test", I see
good things about that in the OBAM world. I'd add though to make the
requirements & test plan/test procedures public so they can be "peer
reviewed".
Lucky
-------------- Original message --------------
From: "Robert L. Nuckolls, III" <nuckollsr(at)cox.net>
>
>
>
> On 15 Sep 2006, at 10:18, Gary Casey wrote:
>
> >
> >
> >
> > Kevin previously wrote:
> >> "If you don't do the test then, someday you may
> >> stumble across that condition when you hadn't planned it. If the
> >> system works as expected, then everything is OK. But if the
system
> >> does not perform, then you may lose the aircraft.
> >
> > Good points, but what "assumptions" do you have in mind that
should
> > be tested?
>
> >
> >
> >
> >Try writing down a list of things your fuel system must do.
Write simple
> >statements and start with the basics - transfer fuel from the
tank to
> >the carb/injector fuel inlet at xx pressure. Write only one
'feature'
> >per statement. Think about what you want to happen in unusual
situations
> >or when components fail. Try not to think about your design.
You are
> >creating a set of "Requirements". Include details - must be
able to get
> >at fuel filter easily for servicing - as well as whole system
issues -
> >system must only need servicing yearly. Define words like
'easily'.
> >
> >There are now two parallel paths to follow. 1) Look at your
design to
> >ensure you have met all of the requirements - be critical (not
always
> >easy). If the design does not meet the requirements then change
the
> >design.
> >
> >2) Develop a verification or test for each requirement. If a
ground test
> >is too difficult you will now understand any risks you are
taking - for
> >example you may decide to trust the pressure gauge manufacturer's
> >calibration (most people do). Minimize flight testing, its
risky and
> >getting good data is more difficult than you think. The best
way is to
> >get someone other than the designer to write the tests (your
wife may
> >be?), but that's not always possible.
> >
> >Be honest with yourself when testing. A second pair of eyes
often helps.
> >Try to keep it all simple.
> >
> >Yours, Pete
>
> Pete, thanks for posting this. An excellent illumination
> of the thought processes behind setting requirements,
> evaluating performance against those requireme nts, a nd
> conducting a failure mode effects analysis (FMEA) from
> which one (1) revises requirements and/or (2) deduces
> Plan-B for each necessary item that fails to perform under
> Plan-A.
>
> With respect to development of requirements, design, testing
> and failure mitigation, I would encourage Listers to publish
> their thoughts here on the List. No doubt some folks will be
> inclined to evaluate your words with too much emphasis on their
> assessment of your lack of skill and knowledge. Expect it to happen
> and know that these postings should be summarily ignored.
>
> However, others are willing and able to assist with step/by/step
> analysis of simple-ideas with the supporting logic needed for
> achieving your design goals. In this manner, we elevate the
> skills and knowledge of all who choose to participate.
>
> The OBAM aviation Skunkwerks ranks among .matro
*
*
|
| | - The Matronics AeroElectric-List Email Forum - | | | Use the List Feature Navigator to browse the many List utilities available such as the Email Subscriptions page, Archive Search & Download, 7-Day Browse, Chat, FAQ, Photoshare, and much more:
http://www.matronics.com/Navigator?AeroElectric-List |
|
|
|
| Back to top |
|
|
luckymacy(at)comcast.net
Guest
|
| Posted: Sun Sep 17, 2006 7:32 am Post subject: Systems Testing |
|
|
Hence the peer review process. Remember, we're not talking big software program where our promotions/pay raises are based on how well we are perceived to be doing. We're talking OBAM with next to no one writing code or making fuel pumps from scratch. Writing your own draft test plan/test procedure is another chance to learn something about yourself and your design. Allows you to think about what you've actually done and find mistakes first. That can be a very satisfying party of this airplane experience. Peer Reviewing your "drafts" will add the sanity check. For us OBAM'ers, it's mostly developing system designs to use off the shelf components and succesfully physically installing them and ensuring they play well with other components.
Just my two cents and I do this for a living too.
[quote]-------------- Original message --------------
From: Deems Davis <deemsdavis(at)cox.net>
[quote] --> AeroElectric-List message posted by: Deems Davis
There's pretty compelling evidence from industry that strongly suggests
that test plans written by the designers are biased to 'prove' the
design. More often then not this is not a conscious but an unconscious
bias. Someone, who is *equally* expert on the subject who is not
invested in the design writing a test plan will uncover more errors and
uncover them earlier. Mike Fagan (at) IBM did some compelling research on
this subject and developed his 'inspection' process for software based
partly upon this principle. Other Quality guru's have done similarly on
manufacturing and other processes.
Deems Davis # 406 < BR>> Panel/wiring
http://deemsrv10.com/
lucky wrote:
> I don't see anything wrong with the designer "writing the test", I see
> good things about that in the OBAM world. I'd add though to make the
> requirements & test plan/test procedures public so they can be "peer
> reviewed".
>
> Lucky
>
> -------------- Original message --------------
> From: "Robert L. Nuckolls, III"
>
> > --> AeroElectric-List message posted by: "Robert L. Nuckolls, III"
> >
> >
> > On 15 Sep 2006, at 10:18, Gary Casey wrote:
> >
> > > --> AeroElectric-List message posted by: Gary Casey
> > >
> > >
> > > Kevin previously wrote:
> > >> "If yo u don't do the test then, someday you may
> > >> stumble across that condition when you hadn't planned it. If the
> > >> system works as expected, then everything is OK. But if the
> system
> > >> does not perform, then you may lose the aircraft.
> > >
> > > Good points, but what "assumptions" do you have in mind that
> should
> > > be tested?
> >
> > >--> Ae roElec tric-List message posted by: "Peter Pengilly"
> > >
> > >
> > >Try writing down a list of things your fuel system must do.
> Write simple
> > >statements and start with the basics - transfer fuel from the
> tank to
> > >the carb/injector fuel inlet at xx pressure. Write only one
> 'feature'
> > >per state ment. Think about what you want to happen in unusual
> situations
> > >or when components fail. Try not to think about your design.
> You are
> > >creating a set of "Requirements". Include details - must be
> able to get
> > >at fuel filter easily for servicing - as well as whole system
> issues -
> > >system must only need servicing yearly. Define words like
> 'easily'.
> > >
> > >There are now two parallel paths to follow. 1) Look at your
> design to
> > >ensure you have met all of the requirements - be critical (not
> always
> > >easy). If the design does not meet the requirements then change
> the
> > >design.
> > >
> > >2) Develop a verification or test for each requirement. If a [b]> > ground test
> > >is too difficult you will now understand any risks you are
> taking - for
> > >example you may decide to trust the pressure gauge manufacturer's
> > >calibration (most people do). Minimize flight testing, its
> risky and
> > >getting good data is more difficult than you think. The best
> way is to
> > >get someone other than the designer to write the tests (your
> wife may
> > >be?), but that's not always possible.
> > >
> > >Be honest with yourself when testing. A second pair of eyes
> often helps.
> > >Try to keep it all simple.
> > >
> > >Yours, Pete
> >
> > Pete, thanks for posting this. An excellent illumination
> > of the thought processes behind set ting r equirements,
> > evaluating performance against those requireme nts, a nd
> > conducting a failure mode effects analysis (FMEA) from
> > which one (1) revises requirements and/or (2) deduces
> > Plan-B for each necessary item that fails to perform under
> > Plan-A.
> >
> > With respect to development of requirements, design, testing
> > and failure mitigation, I would encourage Listers to publish
> > their thoughts here on the List. No doubt some folks will be
> > inclined to evaluate your words with too much emphasis on their
> > assessment of your lack of skill and knowledge. Expect it to happen
> > and know that these postings should be summarily ignored.
> >
> > However, others are willing and able to assist with step/by/step
> > analysis of simple-id eas wi the W [quote][b]
| | - The Matronics AeroElectric-List Email Forum - | | | Use the List Feature Navigator to browse the many List utilities available such as the Email Subscriptions page, Archive Search & Download, 7-Day Browse, Chat, FAQ, Photoshare, and much more:
http://www.matronics.com/Navigator?AeroElectric-List |
|
|
|
| Back to top |
|
|
nuckollsr(at)cox.net
Guest
|
| Posted: Sun Sep 17, 2006 10:15 am Post subject: Systems Testing |
|
|
At 03:31 PM 9/17/2006 +0000, you wrote:
| Quote: | Hence the peer review process. Remember, we're not talking big software
program where our promotions/pay raises are based on how well we are
perceived to be doing. We're talking OBAM with next to no one writing
code or making fuel pumps from scratch. Writing your own draft test
plan/test procedure is another chance to learn something about yourself
and your design. Allows you to think about what you've actually done and
find mistakes first. That can be a very satisfying party of this airplane
experience. Peer Reviewing your "drafts" will add the sanity check. For
us OBAM'ers, it's mostly developing system designs to use off the shelf
components and succesfully physically installing them and ensuring they
play well with other components.
Just my two cents and I do this for a living too.
|
Exactly. Many moons ago, I dreaded being at the front of
the room to present meat for the grinder in a Critical
Design Review.
Years later, I began to look forward to them. It was
a chance to validate the work by fielding the most
probing questions. It became 'fun' when I realized
that irrespective of the outcome of the review, one
of two things would happen: (1) I was able to field
all the questions with solid incorporation of simple-ideas
into an invention the customer wanted and my peers
approved or (2) a bad idea was prevented from going
to production. Win-win all the way around.
It's kinda like biennial flight reviews with a new
instructor. You don't know what they might ask you
to do (my last one was under totally the hood until
200' AGL on final approach). The goal is NOT to demonstrate
repeating all the things you've done before but being
able to do something different while operating within
personal and airframe limits.
That's the real advantage of OBAM aviation. Suggest
anything you want to try with free CDR support.
This atmosphere encourages you to work the problem
from conception to installation knowing that any
biases against the "not invented here syndrome" are
likely to be spotted and avoided.
In TC aviation, we compartmentalize tasks from design
to production so tightly that nobody understands the
whole system. Better to do it all and field the
cabbages and tomatoes than have something slip by
for lack of communication.
Bob . . .
| | - The Matronics AeroElectric-List Email Forum - | | | Use the List Feature Navigator to browse the many List utilities available such as the Email Subscriptions page, Archive Search & Download, 7-Day Browse, Chat, FAQ, Photoshare, and much more:
http://www.matronics.com/Navigator?AeroElectric-List |
|
|
|
| Back to top |
|
|
redeloach(at)fedex.com
Guest
|
| Posted: Mon Sep 18, 2006 6:28 am Post subject: Systems Testing |
|
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> One of the more daunting tasks in problem solving is identifying the problem. All too often, what we see is the result of the problem and we immediately fixate on the result and formulate a plan to "fix" it.
1. Identify and clarify the problem
2. Gather information
3. Evaluate the evidence
4. Consider Alternatives and Implications
5. Choose and Implement the Best Alternatives
There is a great difference between reasoning and rationalizing. We need to ensure we are using reasoning and not just rationalizing (justifying) our actions or action plans.
red :}
lucky wrote: | Quote: | Hence the peer review process. Remember, we're not talking big software program where our promotions/pay raises are based on how well we are perceived to be doing. We're talking OBAM with next to no one writing code or making fuel pumps from scratch. Writing your own draft test plan/test procedure is another chance to learn something about yourself and your design. Allows you to think about what you've actually done and find mistakes first. That can be a very satisfying party of this airplane experience. Peer Reviewing your "drafts" will add the sanity check. For us OBAM'ers, it's mostly developing system designs to use off the shelf components and succesfully physically installing them and ensuring they play well with other components. Just my two cents and I do this for a living too. | Quote: | -------------- Original message --------------
From: Deems Davis <deemsdavis(at)cox.net>
| Quote: |
There's pretty compelling evidence from industry that strongly suggests
that test plans written by the designers are biased to 'prove' the
design. More often then not this is not a conscious but an unconscious
bias. Someone, who is *equally* expert on the subject who is not
invested in the design writing a test plan will uncover more errors and
uncover them earlier. Mike Fagan (at) IBM did some compelling research on
this subject and developed his 'inspection' process for software based
partly upon this principle. Other Quality guru's have done similarly on
manufacturing and other processes.
Deems Davis # 406 < BR>> Panel/wiring
http://deemsrv10.com/
lucky wrote:
> I don't see anything wrong with the designer "writing the test", I see
> good things about that in the OBAM world. I'd add though to make the
> requirements & test plan/test procedures public so they can be "peer
> reviewed".
>
> Lucky
>
> -------------- Original message --------------
> From: "Robert L. Nuckolls, III"
>
> > III"
> >
> >
> > On 15 Sep 2006, at 10:18, Gary Casey wrote:
> >
> > >
> > >
> > >
> > > Kevin previously wrote:
> > >> "If yo u don't do the test then, someday you may
> > >> stumble across that condition when you hadn't planned it. If the
> > >> system works as expected, then everything is OK. But if the
> system
> > >> does not perform, then you may lose the aircraft.
> > >
> > > Good points, but what "assumptions" do you have in mind that
> should
> > > be tested?
> >
> > >
> > >
> > >
> > >Try writing down a list of things your fuel system must do.
> Write simple
> > >statements and start with the basics - transfer fuel from the
> tank to
> > >the carb/injector fuel inlet at xx pressure. Write only one
> 'feature'
> > >per state ment. Think about what you want to happen in unusual
> situations
> > >or when components fail. Try not to think about your design.
> You are
> > >creating a set of "Requirements". Include details - must be
> able to get
> > >at fuel filter easily for servicing - as well as whole system
> issues -
> > >system must only need servicing yearly. Define words like
> 'easily'.
> > >
> > >There are now two parallel paths to follow. 1) Look at your
> design to
> > >ensure you have met all of the requirements - be critical (not
> always
> > >easy). If the design does not meet the requirements then change
> the
> > >design.
> > >
> > >2) Develop a verification or test for each requirement. If a > > ground test
> > > >is too difficult you will now understand any risks you are
|
> > taking - for
> > > >example you may decide to trust the pressure gauge manufacturer's
> > > >calibration (most people do). Minimize flight testing, its
> > risky and
> > > >getting good data is more difficult than you think. The best
> > way is to
> > > >get someone other than the designer to write the tests (your
> > wife may
> > > >be?), but that's not always possible.
> > > >
> > > >Be honest with yourself when testing. A second pair of eyes
> > often helps.
> > > >Try to keep it all simple.
> > > >
> > > >Yours, Pete
> > >
> > > Pete, thanks for posting this. An excellent illumination
> > > of the thought processes behind set ting r equirements,
> > > evaluating performance against those requireme nts, a nd
> > > conducting a failure mode effects analysis (FMEA) from
> > > which one (1) revises requirements and/or (2) deduces
> > > Plan-B for each necessary item that fails to perform under
> > > Plan-A.
> > >
> > > With respect to development of requirements, design, testing
> > > and failure mitigation, I would encourage Listers to publish
> > > their thoughts here on the List. No doubt some folks will be
> > > inclined to evaluate your words with too much emphasis on their
> > > assessment of your lack of skill and knowledge. Expect it to happen
> > > and know that these postings should be summarily ignored.
> > >
> > > However, others are willing and able to assist with step/by/step
> > > analysis of simple-id eas wi the W |
|
| | - The Matronics AeroElectric-List Email Forum - | | | Use the List Feature Navigator to browse the many List utilities available such as the Email Subscriptions page, Archive Search & Download, 7-Day Browse, Chat, FAQ, Photoshare, and much more:
http://www.matronics.com/Navigator?AeroElectric-List |
|
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
Get Email Distribution Too!
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
