Fuzzy trees and forests—Review

Advanced Review

Zenon A. Sosnowski, Łukasz Gadomer

Published Online: Jun 17 2019

DOI: 10.1002/widm.1316

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Abstract Data classification and regression are commonly encountered data analysis problems. Many researchers created multiple tools to deal with these issues. Fuzzy clustering, fuzzy decision trees, and ensemble classifiers such as fuzzy forests are popular tools used for this kind of problems. We would like to describe some interesting, more or less popular, solutions which belong to mentioned areas to show the way they deal with data classification and regression problems. This paper is divided into four parts. In the first part we present the issue of fuzzy clustering, which is one of the most important aspects of fuzzy trees which base on clusters. Some methods of splitting objects into clusters using fuzzy logic are described there. The second part describes different fuzzy decision trees. The way these trees can deal with classification and regression problems is presented. In the third part the issue of forests—ensemble classifiers which consist of fuzzy trees—is described. The last part treats about the way of performing weighted decision making in fuzzy forests. This article is categorized under: Fundamental Concepts of Data and Knowledge > Big Data Mining Technologies > Classification Technologies > Prediction Technologies > Machine Learning

Example division of decision attribute into five contexts using (from left) triangular, Gaussian and trapezoidal membership functions

[ Normal View | Magnified View ]

Example division of decision attribute into four contexts using Gaussian function with context configuration value (from left) 0.6 and 1.6

[ Normal View | Magnified View ]

References

Bezdek,, J., & Pal,, S. K. (1992). In J. C. Bezdek, & S. K. Pal, (Eds.), Fuzzy models for pattern recognition: Methods that search for structures in data. Piscataway, NJ: IEEE Press.
Bezdek,, J. C. (1981). Pattern recognition with fuzzy objective function algorithms. Norwell, MA: Kluwer Academic Publishers.
Bobrowski,, L., & Bezdek,, J. C. (1991). C‐means clustering with the Ll and L? Norms. IEEE Transactions on Systems, Man, and Cybernetics, 21(3), 545–554.
Bonissone,, P. P., Cadenas,, J. M., Garrido,, M. C., & Diaz‐Valladares,, R. A. (2008a). Combination methods in a Fuzzy Random Forest. Paper presented at the IEEE International Conference on Systems, Man and Cybernetics (SMC 2008), Singapore, (pp. 1794–1799).
Bonissone,, P. P., Cadenas,, J. M., Garrido,, M. C., & Diaz‐Valladares,, R. A. (2008b). A Fuzzy Random Forest: Fundamental for design and construction. Paper presented at the Proceedings of the 12th International Conference on Information Processing and Management of Uncertainty in Knowledge‐Based Systems (IPMU`08), Malaga, Spain, (pp. 1231–1238).
Bonissone,, P. P., Cadenas,, J. M., Garrido,, M. C., & Diaz‐Valladares,, R. A. (2010). A Fuzzy Random Forest. International Journal of Approximate Reasoning, 51(7), 729–747.
Breiman,, L. (2001). Random Forests. Machine Learning, 45(1), 5–32.
Breiman,, L., Friedman,, J., Olshen,, R., & Stone,, C. (1984). Classification and regression trees. Monterey, CA: Wadsworth and Brooks.
Dave,, R. N. (1991). Characterization and detection of noise in clustering. Pattern Recognition Letters, 12(11), 657–664.
Dave,, R. N., & Krishnapuram,, R. (1996). M‐estimators and robust fuzzy clustering. In Proceedings of North American fuzzy information processing Berkeley, CA, (pp. 400–404).
Dave,, R. N., & Krishnapuram,, R. (1997). Robust clustering methods: A unified view. IEEE Transactions on Fuzzy Systems, 5(2), 270–293.
Dave,, R. N., & Sen,, S. (1998). Generalized noise clustering as a robust fuzzy c‐M‐estimators model. Paper presented at the 1998 Conference of the North American Fuzzy Information Processing Society (NAFIPS), Pensacola Beach, FL, pp. 256–260.
Dunn,, J. C. (1974). A fuzzy relative of the ISODATA process and its use in detecting compact well‐separated clusters. Journal of Cybernetics, 3, 32–57.
Elyassami,, S., & Idri,, A. (2011). Applying fuzzy ID3 decision tree for software effort estimation. CoRR, abs/1111.0158.
Filev,, D., & Yager,, R. R. (1994). Learning OWA operator weights from data. Paper presented at the Proceedings of the Third IEEE Conference on Fuzzy Systems. IEEE World Congress on Computational Intelligence, Orlando, FL, (Vol. 1, pp. 468–473).
Frigui,, H., & Krishnapuram,, R. (1996). A robust algorithm for automatic extraction of an unknown number of clusters from noisy data. Pattern Recognition Letters, 17(12), 1223–1232.
Frigui,, H., & Krishnapuram,, R. (1997). Clustering by competitive agglomeration. Pattern Recognition, 30(7), 1109–1119.
Frigui,, H., & Krishnapuram,, R. (1999). A robust competitive clustering algorithm with applications in computer vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 450–465.
Gadomer,, L., & Sosnowski,, Z. A. (2016). Fuzzy Random Forest with C‐fuzzy decision trees. Paper presented at the Computer Information Systems and Industrial Management 2016 (LNCS 9842), Vilnius, Lithuania, (pp. 481–492).
Gadomer,, L., & Sosnowski,, Z. A. (2018). Knowledge aggregation in decision‐making process with C‐Fuzzy Random Forest using OWA operators. Soft Computing, 31. https://doi.org/10.1007/s00500-018-3036-x
Groenen,, P. J. F., & Jajuga,, K. (2001). Fuzzy clustering with squared Minkowski distances. Fuzzy Sets and Systems, 120, 227–237.
Groenen,, P. J. F., Kaymak,, U., & van Rosmalen,, J. (2007). Fuzzy clustering with Minkowski distance functions. In J. V. De Oliveira, & W. Pedrycz, (Eds.), Advances in fuzzy clustering and its applications (pp. 53–68). Hoboken, NJ: Wiley‐Blackwell.
Gustafson,, D. E., & Kessel,, W. C. (1978). Fuzzy clustering with a fuzzy covariance matrix. Paper presented at the 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes, San Diego, CA, (pp. 761–766).
Hopner,, F., Hoppner,, F., & Klawonn,, F. (1999). Fuzzy cluster analysis: Methods for classification, data analysis and image recognition. West Sussex, England: Wiley %26 Sons.
Janikow,, C. Z. (1998). Fuzzy decision trees: Issues and methods. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 28(1), 1–14.
Keller,, A. (2000). Fuzzy clustering with outliers. Paper presented at PeachFuzz 2000. 19th International Conference of the North American Fuzzy Information Processing Society (NAFIPS) Atlanta, Georgia, (pp. 143–147).
Klawonn,, F., & Hoppner,, F. (2003). What is fuzzy about fuzzy clustering? Understanding and improving the concept of the fuzzifier. In M. R. Berthold,, H.‐J. Lenz,, E. Bradley,, R. Kruse,, & C. Borgelt, (Eds.), Advances in intelligent data analysis (pp. 254–264). Berlin, Heidelberg: Springer Berlin Heidelberg.
Klawonn,, F., Kruse,, R., & Timm,, H. (1997). Fuzzy shell cluster analysis. In G. Della Riccia,, H.‐J. Lenz,, & R. Kruse, (Eds.), Learning, networks and statistics (pp. 105–119). Vienna, Austria: Springer Vienna.
Krishnapuram,, R., & Keller,, J. M. (1996). The possibilistic C‐means algorithm: Insights and recommendations. IEEE Transactions on Fuzzy Systems, 4(3), 385–393.
Kruse,, R., Doring,, C., & Lesot,, M.‐J. (2007). Fundamentals of fuzzy clustering. In J. V. De Oliveira, & W. Pedrycz, (Eds.), Advances in fuzzy clustering and its applications (pp. 1–30). Hoboken, NJ: Wiley‐Blackwell.
Kumar,, A., Hanmandlu,, M., & Gupta,, H. M. (2013). Fuzzy binary decision tree for biometric based personal authentication. Neurocomputing, 99, 87–97.
Lertworaprachaya,, Y., Yang,, Y., & John,, R. (2014). Interval‐valued fuzzy decision trees with optimal neighbourhood perimeter. Applied Soft Computing, 24, 851–866.
Lichman,, M. (2013). UCI machine learning repository. Retrieved from http://archive.ics.uci.edu/ml
Liu,, X., Feng,, X., & Pedrycz,, W. (2013). Extraction of fuzzy rules from fuzzy decision trees: An axiomatic fuzzy sets (afs) approach. Data and Knowledge Engineering, 84, 1–25.
Liu,, X., & Pedrycz,, W. (2007). The development of fuzzy decision trees in the framework of axiomatic fuzzy set logic. Applied Soft Computing, 7(1), 325–342.
Liu,, Z. G., Pan,, Q., Dezert,, J., & Martin,, A. (2018). Combination of classifiers with optimal weight based on evidential reasoning. IEEE Transactions on Fuzzy Systems, 26(3), 1217–1230.
Macqueen,, J. (1967). Some methods for classification and analysis of multivariate observations. Paper presented at the 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, (pp. 281–297).
Merigo,, J. M., Gil‐Lafuente,, A. M., Yu,, D., & Llopis‐Albert,, C. (2018). Fuzzy decision making in complex frameworks with generalized aggregation operators. Applied Soft Computing, 68, 314–321.
Nefti‐Meziani,, S., & Oussalah,, M. (2007). Inclusion‐based fuzzy clustering. In Advances in fuzzy clustering and its applications (pp. 193–209). Hoboken, NJ: Wiley‐Blackwell.
Pal,, N. R., Pal,, K., & Bezdek,, J. C. (1997). A mixed C‐means clustering model. In Proceedings of 6th international fuzzy systems conference (Vol. 1, pp. 11–21). Piscataway, NJ: IEEE Press.
Pal,, N. R., Pal,, K., Keller,, J. M., & Bezdek,, J. C. (2004). A new hybrid C‐means clustering model. In 2004 IEEE international conference on fuzzy systems (Vol. 1, pp. 179–184). Piscataway, NJ: IEEE Press.
Pedrycz,, W. (1996). Conditional fuzzy C‐means. Pattern Recognition Letters, 17(6), 625–631.
Pedrycz,, W., & Sosnowski,, Z. A. (2005). C‐fuzzy decision trees. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 35(4), 498–511.
Quinlan,, J. R. (1986). Induction of decision trees. Machine Learning, 1(1), 81–106.
Rousseeuw,, P. J., Trauwaert,, E., & Kaufman,, L. (1995). Fuzzy clustering with high contrast. Journal of Computational and Applied Mathematics, 64(1), 81–90.
Sahbi,, H., & Boujemaa,, N. (2005). Validity of fuzzy clustering using entropy regularization. Paper presented the 14th IEEE International Conference on Fuzzy Systems (FUZZ 2005), Reno, NV, (pp. 177–182).
Sardari,, S., Eftekhari,, M., & Afsari,, F. (2017). Hesitant fuzzy decision tree approach for highly imbalanced data classification. Applied Soft Computing, 61, 727–741.
Scariot da Silva,, L. R., Gomide,, F., & Yager,, R. (2007). Fuzzy clustering with participatory learning and applications. In Advances in fuzzy clustering and its applications (pp. 137–153). Hoboken, NJ: Wiley‐Blackwell.
Scholkopf,, B., & Smola,, A. J. (2001). Learning with kernels: Support vector machines, regularization, optimization, and beyond. Cambridge, MA: MIT Press.
Sosnowski,, Z. A. (2012). Decision rules with fuzzy granulation of knowledge (in Polish). Simulation in Research and Development, 3(4), 225–232.
Timm,, H., Borgelt,, C., Daring,, C., & Kruse,, R. (2004). An extension to possibilistic fuzzy cluster analysis. Fuzzy Sets and Systems, 147(1), 3–16.
Timm,, H., & Kruse,, R. (2002). A modification to improve possibilistic fuzzy cluster analysis. Paper presented at the Proceedings of the 2002 IEEE World Congress on Computational Intelligence. 2002 IEEE International Conference on Fuzzy Systems (FUZZ‐IEEE 2002), Honolulu, HI, (Vol. 2, pp. 1460–1465).
Umanol,, M., Okamoto,, H., Hatono,, I., Tamura,, H., Kawachi,, F., Umedzu,, S., & Kinoshita,, J. (1994). Fuzzy decision trees by fuzzy ID3 algorithm and its application to diagnosis systems. Paper presented at the Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference, Orlando, FL, (Vol. 3, pp. 2113–2118).
Vapnik,, V. N. (1995). The nature of statistical learning theory. Berlin, Heidelberg: Springer‐Verlag.
Wang,, X., Liu,, X., Pedrycz,, W., & Zhang,, L. (2015). Fuzzy rule based decision trees. Pattern Recognition, 48(1), 50–59.
Weber,, R. (1992). Fuzzy‐ID3: A class of methods for automatic knowledge acquisition. Paper presented at the 2nd International Conference on Fuzzy Logic Neural Networks, Iizuka, Japan (pp. 265–268).
Wu,, K., & Yang,, M. (2002). Alternative C‐means clustering algorithms. Pattern Recognition, 35, 2267–2278.
Wu,, Z., Xie,, W., & Yu,, J. (2003). Fuzzy C‐means clustering algorithm based on kernel method. Paper presented at the Proceedings of the Fifth International Conference on Computational Intelligence and Multimedia Applications. ICCIMA 2003, Xi`an, China, (pp. 49–54).
Xia,, J., Zhang,, S., Cai,, G., Li,, L., Pan,, Q., Yan,, J., & Ning,, G. (2017). Adjusted weight voting algorithm for Random Forests in handling missing values. Pattern Recognition, 69, 52–60.
Yager,, R. R. (1990). A model of participatory learning. IEEE Transactions on Systems, Man, and Cybernetics, 20(5), 1229–1234.
Zhang,, D., & Chen,, S. (2003a). Clustering incomplete data using kernel‐based fuzzy C‐means algorithm. Neural Processing Letters, 18(3), 155–162.
Zhang,, D., & Chen,, S. (2003b). Kernel‐based fuzzy and possibilistic C‐means clustering. Paper presented at the International Conference on Artificial Neural Networks (ICANN03), Istanbul, Turkey, (pp. 122–125).

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

Fuzzy trees and forests—Review

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox