Present paper extends the novel learning technique of evolutionary support vector machines to $k$-class classification, $k > 2$, by means of hybridization of the inherent evolutionary algorithm with different multi-class support vector machines techniques. The aim of the new paradigm is to evolve, in a simpler and direct manner, the coefficients of the separating surfaces induced by support vector machines. We propose three methods to be integrated in the new approach, i.e. one-against-one, one-against-all and decision directed acyclic graph. Comparison is achieved on the benchmark data set of Fisher's Iris flowers and proves that, for low dimensional problems, the three approaches perform in a similar fashion and are competitive to their canonical counterparts.