jmig
SOH-CM-2024
Over the last ten years or so, we have all seen MSFS models go from $5.00 each to $50.00 and more. With the increase in pricing, have come increased systems integration and model complexity. Interior and exterior visual acuity rivals today real aircraft. As the realism of flight sim models have increased, so too have the expectations of the consumer.
Developers are caught between the customers’ desire for ever better and more exact models, and the increased costs in man hours to program and test these models. It seems too many developers that no matter how realistic and true to the real aircraft a FSX model is, someone will find fault with the model. It is very discouraging to spend months pouring your time and energy into a model, only to have it trashed and dissed by some ‘know it all’ customer.
I have been on both sides of the argument. I have been the ‘know it all’ customer, once ripping a developer for what I considered a very unbelievable flight model. I have also been a beta tester for several different developers and done unofficial testing for two top notch FDE programmers. I thought there might be some interest in blogging what goes on inside a beta testing team before the release of a new model. Maybe it will help us to understand the issues with which developers have to struggle?
Years ago simulated aircraft were much simpler than today. Most switches didn’t work. VCs were primitive, if available. Today, a developer would be roasted in flight sim forums for putting out a model with no VC and nonfunctional cockpit switches. We expect more and the developers are providing more.
Current developers of complex aircraft have gone well beyond the built in features of FSX. Companies like A2A, VRS, Iris, Milviz and others have developed software that runs outside of FSX to simulate advanced features found in their complex aircraft models.
The recently released Milviz F-15E and the VRS F/A-18 are just two examples of aircraft where much of the model consists of programing that is not found within FSX. Another is the Aerosoft F-16. While this affords the consumer the ability to drop bombs, shoot down AI aircraft, or simulate fly by wire aircraft handling (none of which can be found in the FSX SDK) it presents large engineering problems for the developers and their beta teams.
Years ago, beta teams made sure the flight model was realistic, meaning it flew by the published performance numbers from the manufacturer. They checked the visual model for accuracy with the real airplane. If the flight model was good and she looked like the real bird, the model was released. This could take from a week to a month. Today beta testing it is more complex and time consuming.
It begins as soon as a flyable FDE is ready, often before the visual model or VC are ready. The early alpha testing, which can be done using existing aircraft models, checks the raw numbers to the published figures. Are takeoff and approach speeds correct? Climb numbers and fuel consumption are checked. Slowly, the model becomes more and more refined.
In developing the visual model a particular block number may be chosen or a generic version of the model is used. The reason for this it that often the same model’s cockpit can have several variations. You may see different variations within different aircraft block numbers. To make it more confusing, sometimes an upgrade is applied to over a period of time. So, one cockpit of the same model and block number can differ from another. The developer has to choose which one will be represented. Usually, photographs are collected and sorted through looking for the representative cockpit of that particular aircraft model.
Once the visual model and visual cockpit become available to the beta team the controls and systems are checked. Do all the buttons and switches work as published? Do the gauges reflect what is happing in the flight? Do all the cockpit seams fit properly without gaps or rough edges? Are labels and gauge markings correct? And yes! Rivets are checked for accuracy in spacing and visual effect.
In complex aircraft this can be quite a long and difficult process. Military aircraft can be very daunting and some systems are impossible to fully simulate. Sometimes it becomes a matter of subjective decisions as to what to simulate and what to leave out.
Beta testing today is often a matter of compromise and balance. Today, developers try and get the aircraft to perform as close to what the real aircraft does as possible in all flight realms. For a military fighter this can be from slow high AOA flight at sea level to supersonic flight at FL30 or higher. The programmers of FSX never designed the sim to do all of this.
It is easy for a flight model developer to program a model to fly slow approaches well. Or, they can design the model to fly Mach 2 at FL35. Designing it to do both correctly is difficult. They often have to compromise. Compromise can then lead to long forum threads on how the developer’s new model doesn’t climb as per the book, or such and such simulated system doesn’t reflect the real world system correctly.
To get around these limitations some developers have taken to programming advance systems and performance outside of FSX and then integrating these routines into FSX. Sometimes these new routines don’t integrate well. It is here that programmers and beta team members start to pull their hair out. So often, it seems that fixing one thing breaks another.
Back and forth goes the testing. The team reports and the modeler attempts to adjust. As many as three or four versions of the aircraft can be released in a week. Each has to be flown and tested. Eventually the two teams come closer and closer to finding a balance between what is wanted and what can be achieved within FSX and the modeling team’s skills. Eventually, the decision is made that it is good as it will get.
At this point some developers will bring in additional testers. The new testers have fresh eyes. They may see things missed by the original team. So, once again the model goes in for tweaking. Hopefully this last tweaking doesn’t break something else? The final model is teamed up with the extra paints and packaged for sale. The teams take a collective breath and release the new model.
Hopefully, they didn’t miss anything big. More and more often, it seems today that things do get by. The complexity of the models, the scarcity of team members and time to test all contribute to missed items. Then sure as the sun rises in the east, someone will point out the missed item. What is disheartening to the development team is if the entire model is judged based on the one or two items which are incorrect or thought to be incorrect. After months of testing and retesting, some self-appointed expert passes judgment on hundreds of hours of testing. Do you still want to be a beta tester?
I have no illusions that this blog article will keep anal rivet counters from dissing an entire model over some real or perceived inaccuracy. I do hope I have helped fair minded consumers to better understand the process a development team goes through in a model’s development. The price of models has gone up and up. However, we get more for our money today than ever before.
Let’s support the people who support us. Maybe if we judge the entirety of their work, fewer developers will feel the need to build walls and circle the wagons to protect themselves from the arrows?
Developers are caught between the customers’ desire for ever better and more exact models, and the increased costs in man hours to program and test these models. It seems too many developers that no matter how realistic and true to the real aircraft a FSX model is, someone will find fault with the model. It is very discouraging to spend months pouring your time and energy into a model, only to have it trashed and dissed by some ‘know it all’ customer.
I have been on both sides of the argument. I have been the ‘know it all’ customer, once ripping a developer for what I considered a very unbelievable flight model. I have also been a beta tester for several different developers and done unofficial testing for two top notch FDE programmers. I thought there might be some interest in blogging what goes on inside a beta testing team before the release of a new model. Maybe it will help us to understand the issues with which developers have to struggle?
Years ago simulated aircraft were much simpler than today. Most switches didn’t work. VCs were primitive, if available. Today, a developer would be roasted in flight sim forums for putting out a model with no VC and nonfunctional cockpit switches. We expect more and the developers are providing more.
Current developers of complex aircraft have gone well beyond the built in features of FSX. Companies like A2A, VRS, Iris, Milviz and others have developed software that runs outside of FSX to simulate advanced features found in their complex aircraft models.
The recently released Milviz F-15E and the VRS F/A-18 are just two examples of aircraft where much of the model consists of programing that is not found within FSX. Another is the Aerosoft F-16. While this affords the consumer the ability to drop bombs, shoot down AI aircraft, or simulate fly by wire aircraft handling (none of which can be found in the FSX SDK) it presents large engineering problems for the developers and their beta teams.
Years ago, beta teams made sure the flight model was realistic, meaning it flew by the published performance numbers from the manufacturer. They checked the visual model for accuracy with the real airplane. If the flight model was good and she looked like the real bird, the model was released. This could take from a week to a month. Today beta testing it is more complex and time consuming.
It begins as soon as a flyable FDE is ready, often before the visual model or VC are ready. The early alpha testing, which can be done using existing aircraft models, checks the raw numbers to the published figures. Are takeoff and approach speeds correct? Climb numbers and fuel consumption are checked. Slowly, the model becomes more and more refined.
In developing the visual model a particular block number may be chosen or a generic version of the model is used. The reason for this it that often the same model’s cockpit can have several variations. You may see different variations within different aircraft block numbers. To make it more confusing, sometimes an upgrade is applied to over a period of time. So, one cockpit of the same model and block number can differ from another. The developer has to choose which one will be represented. Usually, photographs are collected and sorted through looking for the representative cockpit of that particular aircraft model.
Once the visual model and visual cockpit become available to the beta team the controls and systems are checked. Do all the buttons and switches work as published? Do the gauges reflect what is happing in the flight? Do all the cockpit seams fit properly without gaps or rough edges? Are labels and gauge markings correct? And yes! Rivets are checked for accuracy in spacing and visual effect.
In complex aircraft this can be quite a long and difficult process. Military aircraft can be very daunting and some systems are impossible to fully simulate. Sometimes it becomes a matter of subjective decisions as to what to simulate and what to leave out.
Beta testing today is often a matter of compromise and balance. Today, developers try and get the aircraft to perform as close to what the real aircraft does as possible in all flight realms. For a military fighter this can be from slow high AOA flight at sea level to supersonic flight at FL30 or higher. The programmers of FSX never designed the sim to do all of this.
It is easy for a flight model developer to program a model to fly slow approaches well. Or, they can design the model to fly Mach 2 at FL35. Designing it to do both correctly is difficult. They often have to compromise. Compromise can then lead to long forum threads on how the developer’s new model doesn’t climb as per the book, or such and such simulated system doesn’t reflect the real world system correctly.
To get around these limitations some developers have taken to programming advance systems and performance outside of FSX and then integrating these routines into FSX. Sometimes these new routines don’t integrate well. It is here that programmers and beta team members start to pull their hair out. So often, it seems that fixing one thing breaks another.
Back and forth goes the testing. The team reports and the modeler attempts to adjust. As many as three or four versions of the aircraft can be released in a week. Each has to be flown and tested. Eventually the two teams come closer and closer to finding a balance between what is wanted and what can be achieved within FSX and the modeling team’s skills. Eventually, the decision is made that it is good as it will get.
At this point some developers will bring in additional testers. The new testers have fresh eyes. They may see things missed by the original team. So, once again the model goes in for tweaking. Hopefully this last tweaking doesn’t break something else? The final model is teamed up with the extra paints and packaged for sale. The teams take a collective breath and release the new model.
Hopefully, they didn’t miss anything big. More and more often, it seems today that things do get by. The complexity of the models, the scarcity of team members and time to test all contribute to missed items. Then sure as the sun rises in the east, someone will point out the missed item. What is disheartening to the development team is if the entire model is judged based on the one or two items which are incorrect or thought to be incorrect. After months of testing and retesting, some self-appointed expert passes judgment on hundreds of hours of testing. Do you still want to be a beta tester?
I have no illusions that this blog article will keep anal rivet counters from dissing an entire model over some real or perceived inaccuracy. I do hope I have helped fair minded consumers to better understand the process a development team goes through in a model’s development. The price of models has gone up and up. However, we get more for our money today than ever before.
Let’s support the people who support us. Maybe if we judge the entirety of their work, fewer developers will feel the need to build walls and circle the wagons to protect themselves from the arrows?