The main aim of text categorization is the classification of documents into a fixed number of predefined categories. In text categorization, the dimensionality of the feature vector is usually high. Various approaches have been proposed to reduce the dimensionality of the feature vector while performing automatic text categorization. This paper deals a fast fuzzy self-constructing algorithm that reduces the dimensionality of a feature vector. We also perform automatic categorization of text and hypertext documents using a Support Vector Machines (SVMs) classifier. AS an illustrative example, we considered a set of documents with 15 documents with up to 30 feature words. A fuzzy self-constructing algorithm was used to obtain the reduced number of features. During the training phase the SVM classifier was trained using the reduced set of features. During the decision making phase the SVM classifier was used to classify unknown documents.