premature to develop such a professional standard today. There is no profession yet, it has not yet been established. The big picture, as seen by the Ministry of Labor, still lacks both supply and demand for this profession,” Alexey Kuzmin explains.
Natalia Kosheleva suggests that the ministry’s job was to develop professional standards for basic professions, something that a program and policy evaluation specialist clearly isn’t so far.
“The Ministry of Labor had different expectations at the time, and it had a mission to cover all mass occupations. Ours is definitely not one of those. I think that was the reason. But the standard may indeed emerge at some point, as the profession develops further. When we gain enough visibility for the state to decide we need regulation. It’s like the story of licensing all kinds of activities. To be seen, there must be a lot of you,” says Natalia Kosheleva.
The National Council for Professional Qualifications under the President of the Russian Federation also reviews drafts of professional standards in Russia. Based on the National Council’s conclusions, a decision is made to approve professional standards. As explained by Artem Shadrin, General Director of the National Agency for the Development of Qualifications, when developing a professional standard, it is important not to duplicate existing ones, approved under a similar name. It would be good to get the support of some reputable organizations working in this area, as well as the relevant federal ministry, which would confirm the need for developing such a document.
Once a professional standard has been approved by the Ministry of Labor of Russia, it is then sent to the National Council for Professional Qualifications under the President of the Russian Federation. If approved by the National Council, which represents leading employers’ organizations, such as the Russian Union of Industrialists and Entrepreneurs, the Chamber of Commerce and Industry, trade unions and federal ministries, the standard is then enacted by order of the Ministry of Labor and registered with the Ministry of Justice.
Why does the professional standard story remain unfinished? What could be the reasons?
“First, the Ministry of Labor may see no need for such a professional standard. This is a classic unsolved problem. For example, as for a director per se, there is no such professional standard, and the ministry believes that it is not necessary. Likewise, there is no professional standard for an entrepreneur. On the one hand, this doesn’t seem like a job function; but, on the other hand, it can be considered a type of professional activity. The Ministry of Labor has not yet agreed to develop a professional standard for this activity,” Artem Shadrin says.
He explains it may also happen that an initiative group tasked with developing a professional standard fails to have it approved by the relevant ministry, or the project might not win support from professional associations.
“You cannot get a standard approved without a consensus in the professional community. That’s the challenge: developing a professional standard is a search for touching points. Its development is a long story, but nevertheless, nothing is impossible. As long as there is a desire and support from the relevant ministry, which is always important,” Artem Shadrin adds.
WHAT DOES THE EVALUATION SPECIALIST AS A PROFESSION STRIVE TO GIVE TO THE SOCIETY?
The American philosopher Thomas Schwandt wrote an article in 2017, in which he outlined the need to discuss the ethos of evaluation[20]. That is, the lifestyle of a social group that associates itself with evaluation. According to him, right now, most of the professional community is just talking about competencies and training. Specifically, he writes:
“…Most glaring is the lack of active discussion about what professionalism in evaluation means and what the profession itself seeks to bring to the society.”… “The critical question for the global evaluation community is how that community understands its role in relation to its clients and the society at large.”
What do members of the Russian program and policy evaluation community think about this, and how would they answer the question posed by Thomas Schwandt?
“To me, the profession of an evaluator means objectivity and a competent approach to management decision-making. Because evaluation is based on the principle of triangulation. Before you make any decisions, you have to consider your issue from different perspectives. There is triangulation by attribute, by method, by data source. You cannot make a decision based on any single fact,” Alena Bogomolova reasons.
Irina Efremova-Garth believes that evaluation gives the public a sense of confidence in the quality of management and the appropriate level of effectiveness of the programs implemented by the state, a company, or a nonprofit organization.
“If we talk about the nonprofit sector, the focus is often on how many projects have been supported, how many beneficiaries are covered, and how many grants have been awarded. These are important indicators, but they don’t give us any information about what changes are taking place because of that nonprofit’s work. At best, these numbers only describe the grant manager’s performance. But they do not explain how our lives or the lives of certain groups change for the better,” the expert says.
When you have the evidence base and can show what actions have been taken to address a social problem and what their effect was, this is a highly valuable step, because it helps build confidence in the society.
Irina Sinelina of the International Labor Organization believes that the public benefit provided by evaluation is the accountability in the use of money.
“The ILO, as an international organization, uses taxpayers’ money that comes from different countries. Evaluation is an opportunity to understand how well the funds were spent, and how they can be spent better in the future, how the projects can be improved, and how change can be effected. We collect data, we learn, we study the degree of usefulness of our activities. This knowledge guides our future decisions”, Irina Sinelina explains.
Natalia Kosheleva agrees that evaluation
