Provides conversion between text encodings. More...

#include <qtextcodec.h>

Inheritance diagram for QTextCodec:

Public Member Functions
virtual	~QTextCodec ()

virtual const char *	name () const =0

virtual int	mibEnum () const =0

virtual QTextDecoder *	makeDecoder () const

virtual QTextEncoder *	makeEncoder () const

virtual QString	toUnicode (const char *chars, int len) const

virtual QCString	fromUnicode (const QString &uc, int &lenInOut) const

QCString	fromUnicode (const QString &uc) const

QString	toUnicode (const QByteArray &, int len) const

QString	toUnicode (const QByteArray &) const

QString	toUnicode (const char *chars) const

virtual bool	canEncode (QChar) const

virtual bool	canEncode (const QString &) const

virtual int	heuristicContentMatch (const char *chars, int len) const =0

virtual int	heuristicNameMatch (const char *hint) const

Static Public Member Functions
static QTextCodec *	loadCharmap (QIODevice *)

static QTextCodec *	loadCharmapFile (QString filename)

static QTextCodec *	codecForMib (int mib)

static QTextCodec *	codecForName (const char *hint, int accuracy=0)

static QTextCodec *	codecForContent (const char *chars, int len)

static QTextCodec *	codecForIndex (int i)

static QTextCodec *	codecForLocale ()

static void	deleteAllCodecs ()

static const char *	locale ()

Protected Member Functions
	QTextCodec ()

Static Protected Member Functions
static int	simpleHeuristicNameMatch (const char name, const char hint)

Detailed Description

Provides conversion between text encodings.

By making objects of subclasses of QTextCodec, support for new text encodings can be added to Qt.

The abstract virtual functions describe the encoder to the system and the coder is used as required in the different text file formats supported QTextStream and, under X11 for the locale-specific character input and output (under Windows NT codecs are not needed for GUI I/O since the system works with Unicode already, and Windows 95/98 has built-in convertors for the 8-bit local encoding).

More recently created QTextCodec objects take precedence over earlier ones.

To add support for another 8-bit encoding to Qt, make a subclass or QTextCodec and implement at least the following methods:

const char* name() const: Return the official name for the encoding.
int mibEnum() const: Return the MIB enum for the encoding if it is listed in the IANA character-sets encoding file.

If the encoding is multi-byte then it will have "state"; that is, the interpretation of some bytes will be dependent on some preceding bytes. For such an encoding, you will need to implement

QTextDecoder* makeDecoder() const: Return a QTextDecoder that remembers incomplete multibyte sequence prefixes or other required state.

If the encoding does not require state, you should implement:

QString toUnicode(const char* chars, int len) const: Converts len characters from chars to Unicode.

The base QTextCodec class has default implementations of the above two functions, but they are mutually recursive, so you must re-implement at least one of them, or both for improved efficiency.

For conversion from Unicode to 8-bit encodings, it is rarely necessary to maintain state. However, two functions similar to the two above are used for encoding:

QTextEncoder* makeEncoder() const: Return a QTextDecoder.
QCString fromUnicode(const QString& uc, int& lenInOut ) const;: Converts lenInOut characters (of type QChar) from the start of the string uc, returning a QCString result, and also returning the length of the result in lenInOut.

Again, these are mutually recursive so only one needs to be implemented, or both if better efficiency is possible.

Finally, you must implement:

int heuristicContentMatch(const char* chars, int len) const: Gives a value indicating how likely it is that len characters from chars are in the encoding.

A good model for this function is the QWindowsLocalCodec::heuristicContentMatch function found in the Qt sources.

A QTextCodec subclass might have improved performance if you also re-implement:

bool canEncode( QChar ) const: Test if a Unicode character can be encoded.
bool canEncode( const QString& ) const: Test if a string of Unicode characters can be encoded.
int heuristicNameMatch(const char* hint) const: Test if a possibly non-standard name is referring to the codec.

Definition at line 62 of file qtextcodec.h.

Constructor & Destructor Documentation

QTextCodec::~QTextCodec ( )

virtual

Destructs the QTextCodec. Note that you should not delete codecs yourself - once created they become the responsibility of Qt to delete.

Definition at line 258 of file qtextcodec.cpp.

 {
     if ( !destroying_is_ok )
         qWarning("QTextCodec::~QTextCodec() called by application");
     if ( all )
         all->remove( this );
 }

QTextCodec::QTextCodec ( )

protected

Constructs a QTextCodec, making it of highest precedence. The QTextCodec should always be constructed on the heap (with new), and once constructed it becomes the responsibility of Qt to delete it (which is done at QApplication destruction).

Definition at line 246 of file qtextcodec.cpp.

 {
     setup();
     all->insert(0,this);
 }

Member Function Documentation

bool QTextCodec::canEncode ( QChar ch ) const

virtual

Returns TRUE if the unicode character ch can be fully encoded with this codec. The default implementation tests if the result of toUnicode(fromUnicode(ch)) is the original ch. Subclasses may be able to improve the efficiency.

Definition at line 835 of file qtextcodec.cpp.

 {
     return toUnicode(fromUnicode(ch)) == ch;
 }

bool QTextCodec::canEncode ( const QString & s ) const

virtual

Returns TRUE if the unicode string s can be fully encoded with this codec. The default implementation tests if the result of toUnicode(fromUnicode(s)) is the original s. Subclasses may be able to improve the efficiency.

Definition at line 846 of file qtextcodec.cpp.

 {
     return toUnicode(fromUnicode(s)) == s;
 }

QTextCodec * QTextCodec::codecForContent	(	const char *	chars,
		int	len
	)

static

Searches all installed QTextCodec objects, returning the one which most recognizes the given content. May return 0.

Note that this is often a poor choice, since character encodings often use most of the available character sequences, and so only by linguistic analysis could a true match be made.

See also: heuristicContentMatch()

Definition at line 653 of file qtextcodec.cpp.

 {
     setup();
     QInternalListIterator<QTextCodec> i(*all);
     QTextCodec* result = 0;
     int best=0;
     for ( QTextCodec* cursor; (cursor=i); ++i ) {
         int s = cursor->heuristicContentMatch(chars,len);
         if ( s > best ) {
             best = s;
             result = cursor;
         }
     }
     return result;
 }

QTextCodec * QTextCodec::codecForIndex ( int i )

static

Returns the QTextCodec i places from the more recently inserted, or NULL if there is no such QTextCodec. Thus, codecForIndex(0) returns the most recently created QTextCodec.

Definition at line 343 of file qtextcodec.cpp.

 {
     setup();
     return (uint)i >= all->count() ? 0 : all->at(i);
 }

QTextCodec * QTextCodec::codecForLocale ( )

static

Returns a pointer to the codec most suitable for this locale.

Definition at line 542 of file qtextcodec.cpp.

 {
     if ( localeMapper )
         return localeMapper;
 
     setup();
 
 #ifdef _OS_WIN32_
     localeMapper = new QWindowsLocalCodec;
 #else
     // Very poorly defined and followed standards causes lots of code
     // to try to get all the cases...
 
     char * lang = qstrdup( getenv("LANG") );
 
     char * p = lang ? strchr( lang, '.' ) : 0;
     if ( !p || *p != '.' ) {
         // Some versions of setlocale return encoding, others not.
         char *ctype = qstrdup( setlocale( LC_CTYPE, 0 ) );
         // Some Linux distributions have broken locales which will return
         // "C" for LC_CTYPE
         if ( qstrcmp( ctype, "C" ) == 0 ) {
             delete [] ctype;
         } else {
             if ( lang )
                 delete [] lang;
             lang = ctype;
             p = lang ? strchr( lang, '.' ) : 0;
         }
     }
 
     if( p && *p == '.' ) {
         // if there is an encoding and we don't know it, we return 0
         // User knows what they are doing.  Codecs will believe them.
         localeMapper = codecForName( lang );
         if ( !localeMapper ) {
             // Use or codec disagree.
             localeMapper = codecForName( p+1 );
         }
     }
     if ( !localeMapper || !(p && *p == '.') ) {
         // if there is none, we default to 8859-1
         // We could perhaps default to 8859-15.
         if ( try_locale_list( iso8859_2locales, lang ) )
             localeMapper = codecForName( "ISO 8859-2" );
         else if ( try_locale_list( iso8859_3locales, lang ) )
             localeMapper = codecForName( "ISO 8859-3" );
         else if ( try_locale_list( iso8859_4locales, lang ) )
             localeMapper = codecForName( "ISO 8859-4" );
         else if ( try_locale_list( iso8859_5locales, lang ) )
             localeMapper = codecForName( "ISO 8859-5" );
         else if ( try_locale_list( iso8859_6locales, lang ) )
             localeMapper = codecForName( "ISO 8859-6-I" );
         else if ( try_locale_list( iso8859_7locales, lang ) )
             localeMapper = codecForName( "ISO 8859-7" );
         else if ( try_locale_list( iso8859_8locales, lang ) )
             localeMapper = codecForName( "ISO 8859-8-I" );
         else if ( try_locale_list( iso8859_9locales, lang ) )
             localeMapper = codecForName( "ISO 8859-9" );
         else if ( try_locale_list( iso8859_15locales, lang ) )
             localeMapper = codecForName( "ISO 8859-15" );
         else if ( try_locale_list( tis_620locales, lang ) )
             localeMapper = codecForName( "ISO 8859-11" );
         else if ( try_locale_list( koi8_ulocales, lang ) )
             localeMapper = codecForName( "KOI8-U" );
          else if ( try_locale_list( probably_koi8_rlocales, lang ) )
             localeMapper = ru_RU_hack( lang );
         else if (!lang || !(localeMapper = codecForName(lang) ))
             localeMapper = codecForName( "ISO 8859-1" );
     }
     delete[] lang;
 #endif
 
     return localeMapper;
 }

QTextCodec * QTextCodec::codecForMib ( int mib )

static

Returns the QTextCodec which matches the MIBenum mib.

Definition at line 354 of file qtextcodec.cpp.

 {
     setup();
     QInternalListIterator<QTextCodec> i(*all);
     QTextCodec* result;
     for ( ; (result=i); ++i ) {
         if ( result->mibEnum()==mib )
             break;
     }
     return result;
 }

QTextCodec * QTextCodec::codecForName	(	const char *	hint,
		int	accuracy = `0`
	)

static

Searches all installed QTextCodec objects, returning the one which best matches given name. Returns NULL if no codec has a match closeness above accuracy.

See also: heuristicNameMatch()

Definition at line 626 of file qtextcodec.cpp.

 {
     setup();
     QInternalListIterator<QTextCodec> i(*all);
     QTextCodec* result = 0;
     int best=accuracy;
     for ( QTextCodec* cursor; (cursor=i); ++i ) {
         int s = cursor->heuristicNameMatch(hint);
         if ( s > best ) {
             best = s;
             result = cursor;
         }
     }
     return result;
 }

void QTextCodec::deleteAllCodecs ( )

static

Deletes all the created codecs.

Warning: Do not call this function.

QApplication calls this just before exiting, to delete any QTextCodec objects that may be lying around. Since various other classes hold pointers to QTextCodec objects, it is not safe to call this function earlier.

If you are using the utility classes (like QString) but not using QApplication, calling this function at the very end of your application can be helpful to chasing down memory leaks, as QTextCodec objects will not show up.

Definition at line 81 of file qtextcodec.cpp.

 {
     if ( !all )
         return;
 
     destroying_is_ok = TRUE;
     QInternalList<QTextCodec> * ball = all;
     all = 0;
     ball->clear();
     delete ball;
     destroying_is_ok = FALSE;
 }

QCString QTextCodec::fromUnicode	(	const QString &	uc,
		int &	lenInOut
	)		const

virtual

Subclasses of QTextCodec must reimplement either this function or makeEncoder(). It converts the first lenInOut characters of uc from Unicode to the encoding of the subclass. If lenInOut is negative or too large, the length of uc is used instead.

The value returned is the property of the caller, which is responsible for deleting it with "delete []". The length of the resulting Unicode character sequence is returned in lenInOut.

The default implementation makes an encoder with makeEncoder() and converts the input with that. Note that the default makeEncoder() implementation makes an encoder that simply calls this function, hence subclasses must reimplement one function or the other to avoid infinite recursion.

Reimplemented in QLatin1Codec, QSimpleTextCodec, QTextCodecFromIOD, and QUtf8Codec.

Definition at line 783 of file qtextcodec.cpp.

 {
     QTextEncoder* i = makeEncoder();
     QCString result = i->fromUnicode(uc, lenInOut);
     delete i;
     return result;
 }

QCString QTextCodec::fromUnicode ( const QString & uc ) const

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Definition at line 794 of file qtextcodec.cpp.

 {
     int l = uc.length();
     return fromUnicode(uc,l);
 }

int QTextCodec::heuristicContentMatch	(	const char *	chars,
		int	len
	)		const

pure virtual

Subclasses of QTextCodec must reimplement this function. It examines the first len bytes of chars and returns a value indicating how likely it is that the string is a prefix of text encoded in the encoding of the subclass. Any negative return value indicates that the text is detectably not in the encoding (eg. it contains undefined characters). A return value of 0 indicates that the text should be decoded with this codec rather than as ASCII, but there is no particular evidence. The value should range up to len. Thus, most decoders will return -1, 0, or -len.

The characters are not null terminated.

See also: codecForContent().

Implemented in QLatin1Codec, QSimpleTextCodec, QTextCodecFromIOD, QUtf16Codec, and QUtf8Codec.

int QTextCodec::heuristicNameMatch ( const char * hint ) const

virtual

Returns a value indicating how likely this decoder is for decoding some format that has the given name.

A good match returns a positive number around the length of the string. A bad match is negative.

The default implementation calls simpleHeuristicNameMatch() with the name of the codec.

Reimplemented in QLatin1Codec, QSimpleTextCodec, and QTextCodecFromIOD.

Definition at line 277 of file qtextcodec.cpp.

 {
     return simpleHeuristicNameMatch(name(),hint);
 }

QTextCodec * QTextCodec::loadCharmap ( QIODevice * iod )

static

Reads a POSIX2 charmap definition from iod. The parser recognizes the following lines:

   <code_set_name> name
   <escape_char> character
   % alias alias
   CHARMAP
   <token> /xhexbyte <Uunicode> ...
   <token> /ddecbyte <Uunicode> ...
   <token> /octbyte <Uunicode> ...
   <token> /any/any... <Uunicode> ...
   END CHARMAP

The resulting QTextCodec is returned (and also added to the global list of codecs). The name() of the result is taken from the code_set_name.

Note that a codec constructed in this way uses much more memory and is slower than a hand-written QTextCodec subclass, since tables in code are in memory shared by all applications simultaneously using Qt.

See also: loadCharmapFile()

Definition at line 1279 of file qtextcodec.cpp.

 {
     QTextCodecFromIOD* r = new QTextCodecFromIOD(iod);
     if ( !r->ok() ) {
         delete r;
         r = 0;
     }
     return r;
 }

QTextCodec * QTextCodec::loadCharmapFile ( QString filename )

static

A convenience function for loadCharmap().

Definition at line 1292 of file qtextcodec.cpp.

 {
     QFile f(filename);
     if (f.open(IO_ReadOnly)) {
         QTextCodecFromIOD* r = new QTextCodecFromIOD(&f);
         if ( !r->ok() )
             delete r;
         else
             return r;
     }
     return 0;
 }

const char * QTextCodec::locale ( )

static

Returns a string representing the current language.

Definition at line 1311 of file qtextcodec.cpp.

 {
     static QCString lang;
     if ( lang.isEmpty() ) {
         lang = getenv( "LANG" ); //########Windows??
         if ( lang.isEmpty() )
             lang = "C";
     }
     return lang;
 }

QTextDecoder * QTextCodec::makeDecoder ( ) const

virtual

Creates a QTextDecoder which stores enough state to decode chunks of char* data to create chunks of Unicode data. The default implementation creates a stateless decoder, which is sufficient for only the simplest encodings where each byte corresponds to exactly one Unicode character.

The caller is responsible for deleting the returned object.

Reimplemented in QTextCodecFromIOD, QUtf16Codec, and QUtf8Codec.

Definition at line 726 of file qtextcodec.cpp.

 {
     return new QTextStatelessDecoder(this);
 }

QTextEncoder * QTextCodec::makeEncoder ( ) const

virtual

Creates a QTextEncoder which stores enough state to encode chunks of Unicode data as char* data. The default implementation creates a stateless encoder, which is sufficient for only the simplest encodings where each Unicode character corresponds to exactly one char.

The caller is responsible for deleting the returned object.

Reimplemented in QUtf16Codec.

Definition at line 740 of file qtextcodec.cpp.

 {
     return new QTextStatelessEncoder(this);
 }

int QTextCodec::mibEnum ( ) const

pure virtual

Subclasses of QTextCodec must reimplement this function. It returns the MIBenum (see the IANA character-sets encoding file for more information). It is important that each QTextCodec subclass return the correct unique value for this function.

Implemented in QLatin1Codec, QSimpleTextCodec, QTextCodecFromIOD, QUtf16Codec, and QUtf8Codec.

const char * QTextCodec::name ( ) const

pure virtual

Subclasses of QTextCodec must reimplement this function. It returns the name of the encoding supported by the subclass. When choosing a name for an encoding, consider these points:

On X11, heuristicNameMatch( const char * hint ) is used to test if a the QTextCodec can convert between Unicode and the encoding of a font with encoding hint, such as "iso8859-1" for Latin-1 fonts, "koi8-r" for Russian KOI8 fonts. The default algorithm of heuristicNameMatch() uses name().
Some applications may use this function to present encodings to the end user.

Implemented in QLatin1Codec, QSimpleTextCodec, QTextCodecFromIOD, QUtf16Codec, and QUtf8Codec.

int QTextCodec::simpleHeuristicNameMatch	(	const char *	name,
		const char *	hint
	)

staticprotected

A simple utility function for heuristicNameMatch() - it does some very minor character-skipping so that almost-exact matches score high.

Definition at line 316 of file qtextcodec.cpp.

 {
     // if they're the same, return a perfect score.
     if ( name && hint && qstrcmp( name, hint ) == 0 )
         return qstrlen( hint );
 
     // if the letters and numbers are the same, we have an "almost"
     // perfect match.
     QString h( lettersAndNumbers( hint ) );
     QString n( lettersAndNumbers( name ) );
     if ( h == n )
         return qstrlen( hint )-1;
 
     if ( h.stripWhiteSpace() == n.stripWhiteSpace() )
         return qstrlen( hint )-2;
 
     // could do some more here, but I don't think it's worth it
 
     return 0;
 }

QString QTextCodec::toUnicode	(	const char *	chars,
		int	len
	)		const

virtual

Subclasses of QTextCodec must reimplement this function or makeDecoder(). It converts the first len characters of chars to Unicode.

The default implementation makes a decoder with makeDecoder() and converts the input with that. Note that the default makeDecoder() implementation makes a decoder that simply calls this function, hence subclasses must reimplement one function or the other to avoid infinite recursion.

Reimplemented in QLatin1Codec, QSimpleTextCodec, and QTextCodecFromIOD.

Definition at line 757 of file qtextcodec.cpp.

 {
     QTextDecoder* i = makeDecoder();
     QString result = i->toUnicode(chars,len);
     delete i;
     return result;
 }

QString QTextCodec::toUnicode	(	const QByteArray &	a,
		int	len
	)		const

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Definition at line 803 of file qtextcodec.cpp.

 {
     int l = a.size();
     if( l > 0 && a.data()[l - 1] == '\0' ) l--;
     l = QMIN( l, len );
     return toUnicode( a.data(), l );
 }

QString QTextCodec::toUnicode ( const QByteArray & a ) const

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Definition at line 814 of file qtextcodec.cpp.

 {
     int l = a.size();
     if( l > 0 && a.data()[l - 1] == '\0' ) l--;
     return toUnicode( a.data(), l );
 }

QString QTextCodec::toUnicode ( const char * chars ) const

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Definition at line 824 of file qtextcodec.cpp.

 {
     return toUnicode(chars,qstrlen(chars));
 }

The documentation for this class was generated from the following files:

doxygen-1.8.11/qtools/qtextcodec.h
doxygen-1.8.11/qtools/qtextcodec.cpp

Public Member Functions

Static Public Member Functions

Protected Member Functions

Static Protected Member Functions

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation