Malays J Med Sci. 2022 Jun; 29(3): 133–144. Drug-related problems (DRPs) remain a major health challenge in tertiary health services such as hospitals in Indonesia. These problems are detected and solved using classification systems such as Pharmaceutical Care Network
Europe (PCNE). Therefore, this study aims to obtain a valid and reliable Bahasa Indonesia version of the PCNE. A draft of the Bahasa Indonesia version of the PCNE v9.00 was discussed by four experts from May to August 2020 using the Delphi method. Furthermore, the instrument was assessed for its readability, clarity and comprehensiveness by 46 hospital pharmacists throughout Indonesia. In October 2020, two pharmacists from
Haji General Hospital, Makassar, Indonesia carried out the inter-rater agreement to assess 20 cases where the proportion of coding matches between both raters were observed. The instrument was found to be valid after passing the face and content validity, and the Scale Content Validity Index (S-CVI) value for each PCNE domain was 0.91, 0.89, 0.93, 0.97 and 0.93, respectively. Moreover, there was a fair agreement between the
two raters that ranged between 40%–90%. Also, kappa statistics showed a substantial agreement on the ‘Problems’ and ‘Causes’ domains. The Bahasa Indonesia version of the PCNE v9.00 instrument passed face and content validity as well as inter-agreement to be used in hospital settings. Keywords: drug-related problem, Pharmaceutical Care Network Europe, reliability, translation,
validity Drug-related problems (DRP) are events or circumstances involving drug therapy that occur or potentially interfere with the achievement of desired health outcomes (1). Some of the factors that contribute to the emergence of DRP in patients include inappropriate
prescription, ineffective treatment, underdose, non-compliance, etc (2). In Indonesia, DRP occurred in several chronic diseases such as diabetes (3), kidney
(4) and heart failure (5). Therefore, there is an urgent need for understanding the importance of the pharmacist’s role in identifying, solving and reducing the incidence of DRP in patients
(5). Moreover, the documentation and classification of DRP can help pharmacists to identify and resolve DRP in a patient. Several classification systems, such as APS-Doc, Cipolle, DOCUMENT, Consensus of Granada, Strand Classification and the Pharmaceutical Care Network Europe (PCNE) system, are applied
(6–7). A DRP classification system needs to have an open hierarchical structure with clear definitions for each category described in the instrument to reduce ambiguity or multiple
interpretations when carrying out the coding process (8). Furthermore, the ease of use is also a specific requirement of the DRP classification system, which must be acceptable. Therefore, the DRP classification system needs to be validated before it is widely used
(8–9). Besides having a good validity, an instrument also needs to have a good level of inter-rater reliability, which is a measure of the degree of agreement between two or more raters
(10). This is required to determine the extent to which the raters consistently assign a precise value to each rated item (11). It is essential because the raters need to give the same value in the same
conditions and cases. In this study, the PCNE instrument was selected as the starting point because it is structured, detailed and also identifies the patient’s DRP status based on their problems, causes and interventions. Furthermore, the instrument has several translations including Spanish, Turkish, Croatian, French (12), Slovenia
(7) and German (13). This study, therefore, aims to obtain, validate and determine the inter-rater agreement (percentage agreement and kappa statistics) of Bahasa Indonesia version of the PCNE version
9.00. There was difficulty in downloading the PCNE instrument from their official website; therefore, permission to translate the instrument into Bahasa Indonesia was requested from the PCNE organisation in the Netherlands. The instrument was then downloaded via the PCNE website. Version 9.00 of the PCNE instrument served as the starting point for the
translation and it consisted of five main domains (problem, causes, planned intervention, intervention acceptance, and status of DRP), 24 primary domains and 84 secondary domains (14). The study began with a forward translation of PCNE draft version 9.00 (English version) into Bahasa
Indonesia by two independent sworn translators and the results were separately discussed. Furthermore, the translations were combined by paying attention to the excellent choice of words and pharmaceutical terms. The instrument was again re-translated from Bahasa Indonesia to English by other translators. Similarly, the results were separately discussed. Face ValidityThe combined draft was then given to four experts, including hospital pharmacists and academicians with master and doctoral qualifications. This assisted in the critical review of the translation results and suggestions for improvements to make the instrument easier to use. The process to reach consensus among experts was carried out using the Delphi method (15), which ensures that the expert panels do not know each other and report only to the researcher. A draft of the Bahasa Indonesia version of the PCNE was sent to the expert panels in parallel and they were given time to provide a critical review of the translation. All the critical reviews from each expert panel were then combined and the instrument was refined. The draft was then returned to each expert panel and the process was repeated until they reached a consensus. Content ValidityThis process involved a minimum of 20 clinical pharmacists who work in the hospital as respondents. They were asked to rate the criteria of readability, clarity and comprehensiveness of the instrument using a 5-point Likert scale. Furthermore, the Item Content Validity Index (I-CVI) and the Scale Content Validity Index (S-CVI) were calculated. The I-CVI compares the number of respondents that gave ratings of 3 and above with the total number of respondents. In contrast, the S-CVI is the average of the I-CVI values (16). The following formulae were used to calculate the I-CVI and S-CVI: I-CVI=Number of respondents who give ratingsmore than or equals to 3on Likert scaleTotal respondentsS-CVI= ∑I-CVIN where
The cutoff value for I-CVI and S-CVI are set based on Shrotryia and Dhanda, which is ≥ 0.78 for I-CVI and ≥ 0.8 for S-CVI (17). Inter-Rater AgreementThe inter-rater agreement involved two pharmacists from the Haji Regional General Hospital, Makassar, who conducted a DRP assessment on 20 selected patient cases using validated instruments. These cases were taken from patient medical records using consecutive sampling methods, which met the following eligibility criteria:
Before the test, the two raters were trained separately using five practice cases to familiarise them with the instrument. They were asked to provide a code in the ‘Problem’ and ‘Cause’ domains following the given patient’s DRP case using the Bahasa Indonesia version of PCNE. The coding consistency and chance agreement between the two raters were determined by calculating percentage agreement and kappa statistics. The percentage agreement is the ratio of the number of cases in which both raters gave the same code to the total number of cases. The formula below is used to calculate percentage agreement (18): Percentage agreement=Number of concordant casesTotal number of cases×100% The kappa statistics were carried out using IBM SPSS® version 24 software and its interpretation is showed in Table 1 (18). Table 1Interpretation of kappa values
ResultsThe PCNE classification version 9.00 was translated into Bahasa Indonesia by two sworn translators that did not meet. Furthermore, a reconciliation process was conducted with each translator regarding the translation results, which were then combined. The draft of the translated instrument as shown in Appendix 1 were submitted to the experts for a critical review. After two sessions of discussion with the expert panels, the following changes were incorporated:
A total of 46 hospital pharmacists (Table 2) were recruited from 17 provinces throughout Indonesia (Figure 1) in content validity. The majority of respondents filled 3 and 4 on a 5-point Likert scale, followed by 5, and a few filled 1 and 2. Moreover, the respondents assessed the instrument using four aspects including readability, clarity, ambiguity and comprehensiveness of the instrument. Also, the I-CVI and S-CVI values of each domain’s instrument ranged between 0.85–0.98 and 0.89–0.97, respectively (Table 3). The final version of PCNE after conducting the face and content validity is the Indonesia version shown in Appendix 2. Distribution of the content validity respondents throughout Indonesia Table 2Demographics of respondents who participates in content validity
Table 3I-CVI and S-CVI value on validity content
In addition, the inter-rater agreement involved two pharmacists that worked at Haji General Regional Hospital, Makassar, as raters. The first had 15 years of working experience in the hospital, while the second had 8 years. However, they are not familiar with PCNE instruments, therefore, they were trained using five practice cases. After assessing the DRP cases of 20 patients, the percentage agreement was 90% higher in the ‘problems’ domain for both the primary and secondary domains, respectively. While in the ‘causes’ domain, it was much lower by 60% and 40% on the primary and secondary domains, respectively. Furthermore, kappa statistics were performed to calculate the chance agreement of two raters when identifying the DRP on a case using PCNE. The result showed a significant agreement between the two raters on ‘problems’ domain (κ = 0.615 [95% CI: 0.149, 1.081]; P = 0.003) and ‘Causes’ domain (κ = 0.612 [95% CI: 0.298, 0.910]; P = 0.003). DiscussionThis study is the first to translate, validate and determine the inter-rater agreement of the translated PCNE into Bahasa Indonesia. After the forward (English-Bahasa Indonesia) and backward (Bahasa Indonesia-English) translations, some differences were noticed. These include the changes in word structure, especially in the ‘intervention acceptance’ domain, which is different from the original version because that of Bahasa Indonesia uses conjunctions to make the domain easier for users to understand. According to the suggestions from experts and respondents during the validation process, the changes in the number of word structure compared to the original version before the translation were influenced by the changes in the word structure in Bahasa Indonesia. However, the translation does not differ significantly in the interpretation of the main point of the sentence. Furthermore, the I-CVI and S-CVI values have a high content validity level because they passed Shrotryia and Dhanda’s content validity levels of ≥ 0.78 and 0.8, respectively. However, this value is different from the value reported by Koubaity et al. (12) on the validation of PCNE French. Also, the values of I-CVI and S-CVI were in the range of 0.9–1.0 versus 0.85–0.97 in previous studies. There is also a high consistency in the ‘problems’ domain of the instrument on an inter-rater agreement study. However, the ‘causes’ domain has low consistency, which differs from the results of Koubaity et al. in 2019 and Schindler et al. in 2020 (12–13). Furthermore, these two studies yielded a percentage agreement between 59%–100% and 57.4%–77.3%, respectively. Several factors resulted in the low consistency between the two raters of the ‘causes’ domain. First, this study used a small sample of pharmacists compared to Schindler et al. (13) which considered a total of 32 pharmacists. Second, the variety of codes and the ability to code the case summaries led to different perspectives between the two raters, causing the domain to have low consistency (12). Finally, the raters admitted that it was quite challenging to choose the correct code for a patient’s case, especially in the ‘causes’ domain. Moreover, the two previous studies reported that the raters had difficulty determining the correct code for a given case (13). The kappa statistics showed a high degree of agreement on both ‘problem’ and ‘causes’ domains. The value was higher compared to others such as DOCUMENT (0.53 versus 0.615) (19) and GSASA V2 (0.52 versus 0.615) (20), lower than APS-Doc (0.68 versus 0.615) (21) and similar with the classification developed by the Pharmaceutical Society of Singapore (ILTC DRP Classification System) (0.614 versus 0.615) (8). Furthermore, it is believed that the low value of kappa in this study is because the raters are not too familiar with the instrument. Therefore, using the instrument frequently may increase the value of kappa. This is influenced by the raters’ level of knowledge and experience. This study has certain limitations. First, the instrument does not assess the inter-rater agreement on the ‘planned intervention,’ ‘intervention acceptance,’ and ‘status of DRP’ domains because only secondary data were used. Second, the inter-rater agreement is still limited to only two assessors due to the unfamiliarity of this instrument in daily pharmacy practices in Indonesian hospitals. Furthermore, construct validity, such as convergence to see the instrument’s reliability under different conditions (22), was not performed. Therefore, further studies are suggested to focus mainly on reliability testing by involving more pharmacists and performing the construct validity. ConclusionThe PCNE v9.00, Bahasa Indonesia version has passed content validity and inter-agreement for use in pharmacy practice in both hospitals and academic settings. Further study is suggested to focus mainly on inter-rater reliability tests using more pharmacists to measure the validity of the instruments in various conditions in hospital settings. AcknowledgementsResearchers would like to thank the experts, respondents, and pharmacists in Haji General Regional Hospital, Makassar, who already participate and pour out their thoughts, especially on the instrument’s validation process and inter-rater agreement. Any grant sources did not fund this research. Appendix 1. Bahasa Indonesia version of Pharmaceutical Care Network Europe (PCNE) (after translation)Pharmaceutical Care Network Europe v9.00 versi Indonesia
Appendix 2. Bahasa Indonesia version of Pharmaceutical Care Network Europe (PCNE) (after validity test)Pharmaceutical Care Network Europe v9.00 versi Indonesia
FootnotesEthics of Study This study was ethically approved by the Health Research Ethics Committee of the Faculty of Medicine, University of Indonesia [KET-516/ UN2.F1/ETIK/PPM.00.02.2020] and it was used to collect data for the inter-rater agreement study. To protect the patient’s data and privacy, anonymity was maintained while collecting data and presenting them to raters during the inter-rater agreement study. Conflict of Interest None. Funds None. Authors’ Contributors Conception and design: MAS, RA, SS Analysis and interpretation of the data: MAS Drafting of the article: MAS Critical revision of the article for important intellectual content: RA, SS Final approval of the article: RA, SS Provision of study materials or patients: MAS Statistical expertise: SS Obtaining of funding: MAS References1. Van Mil JWF, Westerlund LOT, Hersberger KE, Schaefer MA. Drug-related problem classification systems. Ann Pharmacother. 2004;38(5):859–867. doi: 10.1345/aph.1D182. [PubMed] [CrossRef] [Google Scholar] 2. Al-Azzam SI, Alzoubi KH, Aburuz S, Alefan Q, Alzayadeen RN. Evaluation of the types and frequency of drug-related problems and the association with gender in patients with chronic diseases attending a primary health care center in Jordan. Int Health. 2016;8(6):423–426. doi: 10.1093/inthealth/ihw026. [PubMed] [CrossRef] [Google Scholar] 3. Zazuli Z, Rohaya A, Adnyana KI. Drug-related problems in type 2 diabetic patients with hypertension: a prospective study. Int J Green Pharm. 2017;11(2):298–304. doi: 10.22377/ijgp.v11i02.1038. [CrossRef] [Google Scholar] 4. Ramadaniati HU, Anggriani Y, Wowor VM, Rianti A. Drug-related problems in chronic kidneys disease patients in an Indonesian hospital: do the problems really matter? Int J Pharm Pharm Sci. 2016;8(12):298–302. doi: 10.22159/ijpps.2016v8i12.15193. [CrossRef] [Google Scholar] 5. Sagita VA, Bahtiar A, Andrajati R. Evaluation of a clinical pharmacist intervention on clinical and drug-related problems among coronary heart disease inpatients: a pre-experimental prospective study at a general hospital in Indonesia. Sultan Qaboos Univ Med J. 2018;18(1):e81–87. doi: 10.18295/squmj.2018.18.01.013. [PMC free article] [PubMed] [CrossRef] [Google Scholar] 6. Basger BJ, Moles RJ, Chen TF. Application of drug-related problem (DRP) classification systems: a review of the literature. Eur J Clin Pharmacol. 2014;70(7):799–815. doi: 10.1007/s00228-014-1686-x. [PubMed] [CrossRef] [Google Scholar] 7. Horvat N, Kos M. Development and validation of the Slovenian drug-related problem classification system based on the PCNE classification V 6.2. Int J Clin Pharm. 2016;38(4):950–959. doi: 10.1007/s11096-016-0320-7. [PubMed] [CrossRef] [Google Scholar] 8. Lim X, Yeo Q, Kng G, Chung W, Yap K. Validation of a drug-related problem classification system for the intermediate and long-term care setting in Singapore. Pharmacy. 2018;6(4):109. doi: 10.3390/pharmacy6040109. [PMC free article] [PubMed] [CrossRef] [Google Scholar] 9. Basger BJ, Moles RJ, Chen TF. Development of an aggregated system for classifying causes of drug-related problems. Ann Pharmacother. 2015;49(4):405–418. doi: 10.1177/1060028014568008. [PubMed] [CrossRef] [Google Scholar] 11. Gisev N, Bell JS, Chen TF. Interrater agreement and interrater reliability: key concepts, approaches, and applications. Res Social Adm Pharm. 2013;9(3):330–338. doi: 10.1016/j.sapharm.2012.04.004. [PubMed] [CrossRef] [Google Scholar] 12. Koubaity M, Lelubre M, Sansterre G, Amighi K, De Vriese C. Adaptation and validation of PCNE drug-related problem classification v6.2 in French-speaking Belgian community pharmacies. Int J Clin Pharm. 2019;41(1):244–250. doi: 10.1007/s11096-0180773-y. [PubMed] [CrossRef] [Google Scholar] 13. Schindler E, Richling I, Rose O. Pharmaceutical Care Network Europe (PCNE) drug-related problem classification version 9.00: German translation and validation. Int J Clin Pharm. 2020;43:726–730. doi: 10.1007/s11096-020-01150-w. [PubMed] [CrossRef] [Google Scholar] 15. Saud A. Master’s thesis. [Depok (West Java)]: University of Indonesia; 2016. Penggunaan teknik Delphi untuk mengidentifikasi standar praktik Pharmaceutical Care di Indonesia. [Google Scholar] 16. Caruso R, Arrigoni C, Groppelli K, Magon A, Dellafiore F, Pittella F, et al. Italian version of dyspnoea-12: cultural-linguistic validation, quantitative and qualitative content validity study. Acta Biomed. 2017;88(4):426–434. doi: 10.23750/abm.v88i4.6341. [PMC free article] [PubMed] [CrossRef] [Google Scholar] 17. Shrotryia VK, Dhanda U. Content validity of assessment instrument for employee engagement. SAGE Open. 2019;9(1):1–7. doi: 10.1177/2158244018821751. [CrossRef] [Google Scholar] 18. Bajpai S, Bajpai R, Chaturvedi HK. Evaluation of inter-rater agreement and inter-rater reliability for observational data: an overview of concepts and methods. J Indian Acad Appl Psychol. 2015;41(3):20–27. [Google Scholar] 19. Williams M, Peterson GM, Tenni PC, Bindoff IK, Stafford AC. DOCUMENT: a system for classifying drug-related problems in community pharmacy. Int J Clin Pharm. 2012;34(1):43–52. doi: 10.1007/s11096-011-9583-1. [PubMed] [CrossRef] [Google Scholar] 20. Maes KA, Tremp RM, Hersberger KE, Lampert ML. Demonstrating the clinical pharmacist’s activity: validation of an intervention oriented classification system. Int J Clin Pharm. 2015;37(6):1162–1171. doi: 10.1007/s11096-015-0179-z. [PubMed] [CrossRef] [Google Scholar] 21. Hohmann C, Eickhoff C, Klotz JM, Schulz M, Radziwill R. Development of a classification system for drug-related problems in the hospital setting (APS-Doc) and assessment of the inter-rater reliability. J Clin Pharm Ther. 2012;37(3):276–281. doi: 10.1111/j.1365-2710.2011.01281.x. [PubMed] [CrossRef] [Google Scholar] 22. Bolarinwa O. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Niger Postgrad Med J. 2015;22(4):195–201. doi: 10.4103/1117-1936.173959. [PubMed] [CrossRef] [Google Scholar] Articles from The Malaysian Journal of Medical Sciences : MJMS are provided here courtesy of School of Medical Sciences, Universiti Sains Malaysia Apa itu range dalam Microsoft Excel?Di Microsoft Excel range juga biasa disebut dengan kumpulan sel. Range dapat berupa dua cell atau lebih. Kumpulan cell yang tergabung dan disebut dengan range bisa dalam satu baris, satu kolom, maupun beberapa baris dan kolom. Bentuk range bisa berupa vertikal, horizontal, campuran, dan ganda.
Rumus apa saja yang ada di excel?Bagi kamu yang ingin mengolah data dengan mudah, berikut adalah rumus excel lengkap yang dapat kamu gunakan dalam pekerjaan.. SUM. ... . 2. AVERAGE. ... . 3. COUNT. ... . 4. COUNTBLANK. ... . COUNTA. ... . 6. SINGLE IF. ... . 7. MULTI IF. ... . 8. COUNTIF.. Apa arti dari rumus AVERAGE?AVERAGE: Fungsi AVERAGE menampilkan nilai rata-rata numerik dalam set data, mengabaikan teks.
Apa arti dari rumus SUM?Fungsi SUM menambahkan nilai. Anda dapat menambahkan nilai individual, referensi sel atau rentang, atau campuran ketiganya. Misalnya: =SUM(A2:A10) Menambahkan nilai di sel A2:10.
|