While earlier generations managed without coaches, leaders today face more pressures than at any other time. They must deal with rapidly changing markets, technologies and workforces, increased financial and legal scrutiny … and more. Top executives who feel that they can handle it by themselves are more likely to burn out, and make poor or no decisions, resulting in significant loss of opportunities, human resources and financial resources.

The CEO’s job is unique from several perspectives: No one else needs to hear the truth more, yet gets it less from employees; no one else is the focus of criticism when things go wrong; no one else is the final decision maker on difficult and often lose-lose decisions; and no one else enjoys the same type of hero-celebrity status and rewards.

According to the Harvard Business Review, two of every five new chief executives fail in their first 18 months on the job. And it appears the main reason for failure has nothing to do with competence, knowledge, or experience, but rather with hubris, ego and a leadership style out of touch with today.

Sydney Finkelstein, author of Why Smart Executives Fail, researched several spectacular failures during a six-year period. He concluded that these chief executives had similar deadly habits chiefly related to unchecked egos. David Dotlich and Peter C. Cairo, in their book, Why CEOs Fail: The 11 Behaviors That Can Derail Your Climb To The Top And How To Manage Them, present 11 cogent reasons why leaders fail, most of which have to do with hubris, ego and a lack of emotional intelligence. Call it overconfidence or ego, but powerful and successful leaders often distrust or feel they don’t need advice.

A study by Kelly See, Elizabeth Wolfe Morrison, and Naomi Rothman, published in Organizational Behavior and Human Decision, concluded one characteristic of powerful and successful leaders is high levels of self-confidence. Unfortunately, the researchers say, the higher the self-confidence, the less likely these leaders are open to advice and feedback. They also note that powerful leaders seldom get useful feedback at their organizations. See and her colleagues contend that the enormous stress leaders are under often produce anxiety, fear and physical illness, which strong leaders are hesitant to divulge over concern judgments may be made about their capacities.

One reason for this leadership crisis may be the gaps between how leaders see themselves and how others see them. Call it self-awareness. These blind spots can be career limiting. The wider the gap, the more resistance to change. It also makes it difficult to create a positive organizational culture where openness and honesty are encouraged.

Paul Michelman, writing in the Harvard Business Review Working Knowledge, says most major companies now make coaching a core of their executive development programs. The belief is one-on-one personal interaction with an objective third party can provide a focus other forms of organizational support cannot. A 2004 study by Right Management Consultants found 86% of companies used coaches in their leadership development program.

Marshall Goldsmith, a high-profile coach to leaders in Fortune 500 companies and author of The Leader of the Future, argues leaders need coaches when “they feel that a change in behaviour — either for themselves or their team members — can make a significant difference in the long-term success of the organization.”

Eric Schmidt, chairman and chief executive of Google, said his best advice to new CEOs is to have a coach. “Once I realized I could trust him [the coach] and that he could help me with perspective, I decided this was a great idea…” he said. While he admits the cost of executive coaches, particularly a good one, is not cheap, he adds “compared to the decisions CEOs make, money is not the issue.”

“If you have a new perspective, if you feel better with your team, the board and the marketplace, then you have received real value,” Schmidt says.

“Executives who rise to the C-suite do so largely based upon their ability to consistently make sound decisions. However while it may take years of solid decision-making to reach the boardroom it often times takes one bad decision to fall from the ivory tower. The reality is that in today’s competitive business world an executive is only as good as his/her last decision, or their ability to stay ahead of contemporaries and competitors,” Mike Myatt writes in an article, The Benefits of a Top CEO Coach.

Despite its popularity, many senior executives are reluctant to report they have a coach, says Jonathan Schwartz, former president and chief executive of Sun Microsystems, who had an executive coach.

While board members can be helpful, most leaders shy away from talking to the board about their deepest uncertainties, argues John Kador, in CEO Magazine. Other CEOs can lend an ear, but there are barriers to complete honesty and trust. “No one in the organization needs an honest, close and long-term relationship with a trusted advisor more than a CEO,” writes Kador, who reports conversations with several high-profile CEOs.

The much asked question about coaching is its return on investment. The majority of studies including one by Joy McGovern and her colleagues at  research firm Manchester, indicate that executives who received coaching valued the service between $100,000 and $1-million. Joyce Russell, the Dean of the Robert H. Smith School of Business at the University of Maryland contends that assigning a coach to an executive is nowviewed as a privilege and a sign the organization values the executive’s contributions and is willing to invest money in his or her growth and development.

In my Financial Post articles, Top Dogs are Lonely,The Second Fastest Growing Profession and The Seven Deadly Habits of CEOs, I outlined what is now common practice for chief executives — hiring a personal executive coach — and how it helps them perform better. Professional executive coaches can help leaders improve performance, reduce or eliminate their blind spots and be open to constructive feedback, not only reducing the likelihood of failure, and premature burnout but also providing an atmosphere in which the executive can express fears, failures and dreams.

Ray Williams is President of Ray Williams Associates, a company based in Vancouver providing leadership training and executive coaching services. He can be reached at ray@raywilliamsassociates.com