An encyclopedia defines a security protocol as “a sequence of operations that ensure protection of data. Used with an underlying communication protocol, it provides secure delivery of data between two parties.”
We use security protocols in everyday computing. For example, when we use our domain credentials to login to a Microsoft Windows environment, we use the Kerberos (or the NTLM protocol in the earlier versions of Windows) indirect authentication protocol. When we conduct an e-commerce transaction using a secure e-commerce site, we use various encryption protocols, integrity protocols, and so on. Of course, there are other types of security protocols – authentication protocols, access-control (or authorization protocols), encryption protocols, key management protocols, key distribution protocols, and so on.
In day-to-day computing, we use standard security protocols because they offer us significant security benefits.
For one, experts design standard security protocols. Once designed, these are reviewed and re-reviewed by experts on standards organizations (e.g. IEEE, W3C, IETF). New protocols are subjected to security threat-modeling analysis to ensure that they offer protection against commonly known attack patterns. When these protocols are deployed in the field, their security is monitored, and over time their security kinks are worked out.
In addition, when these standard protocols become (eventually) insecure, more secure versions of protocols are made available (TLS 1.1/1.2 to replace TLS 1.0) or new protocols are designed to replace the aging ones (e.g. AES replaced aging DES/3DES).
Then there are weaknesses associated with the use of standard protocols as well. Since all the knowledge pertaining to a standard protocol design / implementation is in the public domain, there is a greater likelihood of finding 0day exploits. Additionally, when an issue discovered, it is widely known and exploitable in a very short period of time. The Heartbleed bug is a good example  of this. Finally, tools are widely available to exploit any older and unpatched versions of the software libraries/packages that are known to implement the standard protocol incorrectly.
In the world of embedded devices there are a few reasons why people may choose to use proprietary protocols:
Whatever the reasons are, when we design our own proprietary protocols, we need to worry about many security issues.
For example, if we are designing a custom cryptographic protocol (a really really bad idea), then we need to worry about whether our protocol is provably secure? If not, then is our proprietary protocol secure against passive attacks (CPA-secure) and/or active attacks (CCA-secure)?
If we are designing a distributed proprietary protocol (say an custom authentication protocol or a custom key distribution protocol), then we need to worry many other security issues such as:
Cigital conducts reviews of thick clients / embedded systems / mobile applications in the healthcare industry, in the gaming industry, and in the financial industry (among others). In our reviews, we commonly find serious design-level and implementation-level issues in proprietary security protocols implementations. In our experience, it is very difficult to get proprietary protocol designed and implemented correctly. Our security analysts are routinely able to find serious vulnerabilities in such protocols. A common example of such proprietary security protocol code is X.509 certificate validation. While most commonly used browsers do a good job of validating server-side X.509 certificates, we routinely find many issues when mobile applications (which are responsible for certificate validation) implement such validation.
Cigital’s recommendation is that we should as much as possible use standard security protocols. If we must use a proprietary security protocol, then we should try to use such a protocol in conjunction with a standard protocol (e.g. implementing proprietary encryption or encoding scheme over a secure TLS link, or the use of custom white-box cryptography controls in conjunction with other well-known obfuscation controls). If a proprietary security protocol is used exclusively, then its design and implementation must be thoroughly assessed for security vulnerabilities.
Chandu Ketkar, Technical Manager, has been with Cigital since 2009. His vast security expertise includes architecture security, secure design assessment automation, medical device and systems security, cryptography, mobile application security, maturity models and software security initiatives (SSI). Chandu also has over 25 years of experience building software and says if he knew back then what he knows now, he would have built it a lot safer. He is a member of the AAMI and is actively engaged in creating an automation tool to scale architecture/design risk assessments. When he’s not building code for medical devices, Chandu relaxes by singing and listening to music.
Words of Security Wisdom
Software developers and software security experts are part of the same ecosystem. They depend on each other to build secure software systems. Security experts need developers as “change agents” to build secure software. Software developers and architects need security experts to educate and advise them on how to ‘Build Security In.”
Cigital is one of the world’s largest application security firms. We go beyond traditional testing services to help our clients find, fix and prevent vulnerabilities in the applications that power their business.
Our experts also provide remediation guidance, program design services, and training that empower you to build and maintain secure applications.
Gary McGraw discusses the security risks of dynamic code
The Cathedral and the Bazaar of Software Security Vulnerabilities
Serving Resources Over SSL With CSP Upgrade-Insecure-Requests
Integrating Touch ID into your iOS applications
The Security Risks of Dynamic Code @cigitalgem | sws.ec/1MXbRt1 pic.twitter.com/oERGOxBFdf
August 27, 2015 11:13 am
A Deficit In Security Spending Has Led To A Massive Security Debt via @stiennon @ForbesTech | sws.ec/1WUz8yS
August 27, 2015 10:34 am
How secure is the Hybrid Cloud? via @CIOonline | sws.ec/1WUtU6j #Appsec pic.twitter.com/aeuVOrGd9e
August 27, 2015 9:50 am
4 Security Metrics That Matter via @FYRashid @CarolineWMWong @CIOonline sws.ec/1Jmjk0P pic.twitter.com/wdyn3aJFOq
August 26, 2015 11:43 am
#APPSEC Opportunities: Our Recruiting team is coming to your campus! Stay current here sws.ec/1KkiPGo pic.twitter.com/6t0EcCzWhL
August 26, 2015 11:18 am